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TOWARD  A METHODOLOGY  FOR  MAN -MACHINE  FUNCTION 

ALLOCATION  IN  THE  AUTOMATION  OF  SURVEILLANCE  SYSTEMS 

VOLUME  I : SUMMARY 

CHAPTER  1 

INTRODUCTION  AND  OVERVIEW 

This  is  Volume  I of  a two-volume  report  on  a study  con- 
cerned with  the  performance  implications  of  various  degrees 
of  automation  in  surveillance  systems.  The  study  was  stimu- 
lated by  a growing  need  for  systematic  feedback  from  operating 
systems  to  the  designers  of  future  systems.  The  need  for  such 
feedback  was  felt  to  be  especially  acute  in  the  class  of  sys- 
tems generally  known  as  information  processing  systems,  of 
which  surveillance  systems  are  an  example.  It  was  hoped  that 
the  study  would  generate  information  that  could  aid  system 
designers  in  making  performance/cost  trade-off  decisions  in 
future  surveillance  systems. 

It  was  felt  that  the  most  significant  feature  of  current 
and  future  information  processing  systems  is  the  extent  to 
which  they  incorporate  computers  and  other  forms  of  automa- 
tion. A trend  toward  more  and  more  dependence  on  automation 
has  been  especially  evident  in  recently  developed  surveillance 
systems.  Both  the  cost  and  performance  implications  have 
proved  to  be  extensive,  and  there  have  been  some  major  disap- 
pointments in  how  effectively  some  functions  can  be  automated. 

, In  some  kinds  of  systems,  the  performance  improvement 

due  to  automation  has  been  quite  dramatic.  For  example, 
automatic  processing,  storing,  and  distribution  of  informa- 
tion in  tactical  da^a  or  command-and-control  systems  can 
, immensely  facilitate  the  making  of  timely  and  valid  decisions. 

Automation  is  especially  effective  in  such  systems  because, 
in  general,  human  beings  are  severely  limited  in  the  informa- 
tion rate  they  can  handle  and,  consequently,  often  become 
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overloaded.  Also,  the  nature  of  most  of  the  information  in 
tactical  data  systems  is  such  that  it  is  not  degraded  by  auto- 
matic processing  and  distribution.  In  contrast,  automation 
techniques  can  often  lead  to  severe  performance  degradation 
when  they  are  applied  to  complex  recognition  and  decision- 
making functions. 

Most  system  functions  lie  somewhere  between  these  extremes 
Often,  the  information  processing  load  is  sufficiently  high 
that  automation  is  tempting.  However,  it  may  be  clear  that 
eome  performance  degradation  will  be  incurred.  Just  how  much 
degradation  to  expect  is  seldom  easy  to  predict. 

It  was  our  purpose  to  address  this  problem  in  surveillance 
systems  using  four  sources  of  information:  (1)  experience  with 

automation  in  several  large-scale  surveillance  systems  during 
the  course  of  their  development  and  trial,  (2)  an  analysis  of 
what  is  known  about  how  man  functions  as  an  information  pro- 
cessor, (3)  test  data  on  how  well  man  performs  various  tasks 
associated  with  the  operation  of  surveillance  systems,  and  (4) 
data  from  experiments  specifically  designed  to  clarify  the 
effects  of  function  allocation  between  man  and  computer  in 

surveillance  systems. 

A good  deal  of  this  study  was  retrospective  in  the  sense 
that  we  had  to  depend  on  past  histories  of  successes  and  fail- 
ures of  automation  in  surveillance  systems  at  various  stages 
of  development.  And  we  are  certainly  not  in  a position  to 
forecast  how  the  future  state-of-the-art  in  automation  will 
cope  with  some  of  the  difficulties  identified  during  the  course 
of  this  study;  but  we  can  assert  the  importance  of  closing  the 
feedback  loop  from  operating  systems  to  the  designers  of  future 
systems.  We  hope  that  the  material  in  these  two  volumes  will 
help  serve  this  purpose  with  respect  to  both  machines  and 
their  human  operators.  There  is  still  a good  deal  of  uncer- 
tainty about  both.  The  following  quotation,  which  was  made  a 
decade  ago,  still  retains  the  clear  ring  of  truth: 
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Those  of  us  who  are  in  the  human  sciences  often 
feel  insecure  and  apologetic  about  our  own 
science  and  subject  matter.  We  often  feel  that 
it  is  our  ignorance  about  human  behavior  which 
is  responsible  for  the  difficulties  we  have  in 
making  decisions  about  man-machine  system  design. 

J.f  you  have  ever  worked  closely  with  systems 
engineers,  however,  you  soon  come  to  realize 
that  there  is  much  uncertainty  on  the  machine 
side  of  the  equation  as  well.  In  the  first 
place,  there  is  no  such  thing  as  a system  design 
process,  precise  and  specifiable....  Instead, 
systems  designers  often  proceed  with  much  trial- 
and-error  just  as  anyone  else  does.  In  addition, 
although  I realize  it  is  dangerous  to  make  a 
generalization  about  this  point,  I have  the 
feeling  that  engineers  are  sometimes  overly 
optimistic  in  their  predictions  about  what  they 
can  do  with  their  machines.  As  a result,  the 
final  products  frequently  fall  short  of  what 
had  been  anticipated  for  them.  (Chapanis,  1965) 

There  is  still  much  ignorance  about  human  behavior,  there 
is  still  much  uncertainty  on  the  machine  side  of  the  equation, 
there  is  still  no  such  thing  as  a system  design  process,  and 
engineers  are  still  sometimes  overly  optimistic  about  what 
they  can  do  with  their  machines.  Despite  substantial  engineer- 
ing efforts  to  automate  most  surveillance  system  tasks,  exten- 
sive automation  is  conspicuously  absent  from  today's  operational 
surveillance  scene.  A large  proportion  of  these  efforts  have 
been  focused,  by  several  different  engineeiing  groups,  in 
several  different  ways,  upon  certain  crucial  surveillance  sys- 
tem functions  which  continue,  obstinately,  to  resist  successful 
automation.  It  now  appears  to  many  that  successful  automation 
of  these  functions  is  not  just  around  the  corner,  and  the  time 
appears  propitious  for  a serious  reconsideration  of  man-machine 
function  allocation  in  the  automation  of  surveillance  systems. 

We  say  "propitious,"  implying  a favorable  condition,  not 
because  things  have  been  going  well;  quite  to  the  contrary. 

It  is  the  heer  magnitude  and  costliness  of  the  problems  which 
have  been  encountered  that,  perhaps,  will  provide  the  necessary 
impetus  for  the  substantial  interdisciplinary  efforts  that  will 
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be  essential  to  resolve  the  function  allocation  problems  we 
are  now  clearly  confronted  with.  Many  years  ago,  human  fac- 
tors specialists  could  specify  with  a reasonable  degree  of 
precision  how  well  each  of  several  surveillance  system  functions 
and  tasks  could  be  executed  by  men,  but  it  was  not  then  possible 
for  engineers  to  demonstrate  corresponding  performance  levels 
for  machines.  In  the  absence  of  empirical  evidence,  a good 
deal  of  optimism  prevailed  regarding  automation.  Today,  the 
strengths  and  weaknesses  of  machines,  as  well  as  men,  are  more 
clear;  and  while  it  is  obvious  that  we  will  not  want  to  cling 
to  "totally"  manual  systems,  the  extensive  degree  of  automation 
anticipated  by  some  cannot  be  regarded  as  a viable  alternative 
for  "next  generation"  systems,  either.  Thus,  human  factors 
specialists  and  systems  engineers  must  now  undertake,  together, 
to  move  into  those  uncertain  regions  where  man  and  machine 
work  as  more  nearly  equal  partners,  executing  tasks  apportioned 
in  various  permutations,  so  that  it  may  be  determined  which 
configurations  bring  about  the  synergistic  man-machine  relation- 
ships that  must  comprise  the  "next  generation"  systems. 

We  do  not  believe  the  human  factors  and  systems  engineer- 
ing disciplines  are  totally  prepared  for  the  function  alloca- 
tion problems  that  now  must  be  faced.  We  have  entitled  this 
report  "Toward  a Methodology..."  because  it  is  not  a "cookbook," 
with  convenient  recipes  for  every  (or  any)  specific  function 
or  task  allocation.  Indeed,  the  complete  "cookbook"  of  function 
allocation  recipes  would  probably  take  longer  to  develop  than 
the  totally  automated  surveillance  system,  therefore  rendering 
it  moot;  but  neither  of  these  remote  developments  should  con- 
cern the  surveillance  community  faced  with  solving  problems 
now . 

We  have  set  down  in  the  two  volumes  of  this  report  a 
perspective  of  the  function  allocation  process  as  we  see  it 
affecting  the  "next  generation"  of  systems.  Some  of  this  in- 
formation, we  hope,  will  make  the  task  of  the  reader  seriously 
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involved  with  specific  function  allocation  problems  easier, 
and  we  will  have  achieved  an  important  goal  if  this  report 
will  provide  him  some  guidance  regarding  the  precise  nature 
of  the  information  gathering  that  he  will  almost  certainly 
have  to  undertake  himself. 

Classified  information,  which  comprises  much  of  the  spe- 
cific information  in  Volume  II,  has  been  omitted  from  Volume  I 
to  permit  a broader  dissemination  of  summary  information,  with 
the  result  that  some  of  the  viewpoints  expressed  in  this  volume 
may  seem  unsupported.  We  hope  that  the  interested  reader  will 
also  turn  to  Volume  II,  the  contents  of  which  are  briefly  out- 
lined at  the  end  of  this  volume. 

The  following  chapter  outlines  a general  functional 
taxonomy  of  surveillance  systems,  together  with  our  conclusions 
concerning  the  merits  and  problems  associated  with  allocating 
each  function  to  man  or  computer.  Volume  II  also  contains  the 
taxonomy,  but  with  some  examples  that  are  classified,  and  a 
somewhat  extended  discussion  of  each  function.  Chapter  3 
presents  a model  of  man  as  an  information  processor  in 
surveillance  systems;  Chapter  4 presents  some  information  on 
the  variable  performance  of  man  in  surveillance  systems;  and 
Chapter  5 is  concerned  with  the  measurement  of  man-machine 
performance  in  surveillance  systems.  Each  of  these  chapters 
has  its  more  detailed  counterpart  in  Volume  II. 
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CHAPTER  2 


MAN-MACHINE  FUNCTION  ALLOCATION  IN 
NEW  SYSTEM  DESIGN 

One  of  the  products  of  this  study  is  a generalized  tax- 
onomy of  surveillance  system  functions.  We  shall  discuss  each 
of  these  functions,  breaking  them  down  into  elements  we  shall 
call  "tasks."  The  reader  should  note  that  we  use  the  term 
"task"  in  a special  sense  that  shall  be  presently  evident, 
and  that  the  term  has  been  used  by  different  authors  to  mean 
different  things.  Each  of  the  taxonomy  functions  is  described 
below.  A rating  is  given  to  each  task  on  a scale  from  1 to  9, 
which  reflects  our  conclusions  concerning  the  necessity  of 
retaining  operator  involvement  in  that  task  versus  the  alter- 
native of  full  automation.  A high  Operator  Involvement  Rating 
(7,  8,  or  9)  is  assigned  to  a task  that  almost  certainly  re- 
quires extensive  roles  for  the  operator  at  this  time,  either 
because  extreme  difficulties  have  been  encountered  in  previous 
attempts  to  automate  such  tasks  or  because  potential  sources 
of  difficulty  can  readily  be  foreseen.  A low  Operator  Involve- 
ment Rating  (1,  2,  or  3)  is  associated  with  tasks  that  clearly 
should  be  automated  because  of  man’s  limited  information  pro- 
cessing capabilities,  computing  capabilities,  or  fallible 
memory,  and  because  it  has  been  demonstrated  that  machines 
can  handle  those  tasks  efficiently.  Intermediate  OIRs  are 
given  to  those  tasks  where  human  information  processing  cur- 
rently plays  a critical  role  but  where  computer-aiding  may 
nonetheless  be  a feasible  and  desirable  system  characteristic. 

We  had  some  misgivings  about  assigning  simple  unidimen- 
sional scores  to  the  admittedly  very  complex  considerations 
of  function  allocation.  We  nevertheless  did  so  because  we 
feel  these  numbers  do  convey  in  a very  generalized  manner  a 
convenient  index  to  the  very  different  degrees  of  risk  in 
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making  the  assignment  of  various  functions  to  men  ot  machines. 
We  readily  admit  that  there  may  be  considerable  room  for  argu- 
ment about  the  different  values  assigned;  this  satisfies  one 
of  our  objectives  because  we  feel  such  arguments  may  be  useful. 

The  reader  will  find  a rationale  for  each  rating  in  the 
function  descriptions  that  follow  Table  2.1.  The  discussion 
that  follows  is  general  and  somewhat  abstract.  The  interested 
reader  is  urged  to  consult  Volume  II,  in  which  the  discussion 
is  more  lengthy  and  is  enhanced  by  examples  drawn  from  specific 
(classified)  surveillance  systems. 

TABLE  2.1 

SURVEILLANCE  SYSTEM  FUNCTION  TAXONOMY 

Operator 
I n vol  vement. 
Rating  (OIK) 

1 . Memo  ry 

1.1  Goal  information  5 

1.2  Function  algorithms  5 

1.3  Stimulus  data  2 

1.4  Non-stimulus  data  5 

2.  Executive  Control 

2.1  Comprehension  of  the  current 

situation  5 

2.2  Interpretation  of  goals  5 

2.3  Interpretation  of  past  situations  5 

2.4  Hypothesis  generation  and  prediction  5 

2.5  Effecting  appropriate  control 

responses  • 5 

2.6  Hypothesis  testing  and  iteration  5 

3.  Attention  Selection 


3.1 

Search  initiation 

6 

3.2 

Sensor  placement 

6 

3.3 

Spatial  coverage  and  spatial 
resolution  selection 

5 

3.4 

Selection  of  other  parameters 

5 

8 


F 


w 


k 

\ 


Operator 
I nvol vement 
Rating  (OIR) 


4. 

Stimulus  Transmission 

4.1 

Pre-sensor  transmission 

N/A 

4.2 

Post-sensor  transmission 

1 

5. 

Stimulus  Processing 

5.1 

Beam  forming 

1 

5.2 

Further  processing 

1 

6. 

Detection 

6.1 

Signal  detection 

0 

6.2 

Transient  detection 

8 

6.3 

Track  detection 

3 

7. 

Feature  Extraction/Association 

7.1 

Signal  parameter  estimation  (i.e., 
extraction  of  lowest-order  features) 

6 

7.2 

Screeni ng 

6 

7.3 

Early  classification  based  on  espe- 
cially distinctive  characteristic 

lower-order  features 

2 

7.4 

Association  of  lower-order  features 
into  appropriate  higher-order  features 

8 

7.5 

Alerted  searching  for  undetected 
lower-order  features 

2 

7.6 

Association  of  higher-order  features 
with  targets 

8 

7.7 

Early  classification  based  on  espe- 
cially distinctive  characteristic 

higher-order  features 

2 

8. 

Feature  Space  Transformation 

8.1 

Combination  of  features  to  reduce 
dimensi onal i ty 

5 

8.2 

Combination  of  features  to  maximize 
discrimi nabi 1 i ty 

5 

8.3 

Coordinate  transformation 

1 

9, 

Target 

Local izati on 

9.1 

Single-sensor  fixing 

2 

9.2 

Multiple-sensor  fixing 

2 

9.3 

T racki ng 

2 
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Operator 
I nvol vement 
Rating  (OIR) 


10.  Target  Motion  Analysis 

10.1  Inference  of  motion  based  on 

extracted  stimulus  features  2 

10. 1 Posi tion-versus  time  analysis  2 

10.3  T rack-a i ding  2 

11.  Classification 

11.1  Determination  of  stimulus  source 

likelihood  estimates  5 

11.2  Alerted  searching  for  undetected 
signals  typically  related  to 

inferred  stimulus  sources  2 

11.3  Determination  of  stimulus  source 

configuration  likelihood  estimates  4 

11.4  Determination  of  operating 

behavior  likelihood  estimates  7 

11.5  Determination  of  target  class 

likelihood  estimates  3 

11.6  Alerted  searching  for  undetected 
signals  typically  related  to 

inferred  target  classes  3 

11.7  Determination  of  target  class 

a priori  probabilities  4 

11.8  Determination  of  target  class 

a posteriori  probabilities  2 

11.9  Determination  of  classification 

decision  risk  functions  3 

11.10  Determination  of  an  appropriate 

decision  rule  2 

11.11  Classification  decision  making  3 

12.  Communication 

12.1  System  coordination  3 

12.2  System  output  3 

12.3  System  input  3 

13.  Learning 

13.1  Strategic  and  tactical  intelligence  5 

13.2  Technical  intelligence  5 

13.3  System  operational  characteristics  5 

13.4  Training  5 
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1.  Memory.  Memory  is  essential  for  even  the  sim- 
plest system  to  operate.  In  simple  systems,  memory 
might  be  regarded  as  the  hard  wiring  which  causes 
the  unit  to  behave  as  it  docs.  In  larger  systems, 
memory  can  be  widely  distributed,  in  many  different 
forms,  throughout  the  system.  It  serves  as  a repos- 
itory for  "goal"  information,  to  assist  the  execu- 
tive function  in  bringing  about  goal -di rected  system 
behavior;  for  the  storage  of  function  algorithms, 
which  can  include  all  the  functions  on  our  list; 
for  the  storage  of  stimulus  data,  upon  which  the 
function  algorithms  work;  and  for  the  storage  of 
non-stimulus  data,  which  are  many  in  kind  and  number 
and  which  are  necessary  to  support  the  operation  of 
the  system.  Crucial  factors  of  the  memory  function 
include  form,  capacity,  storage  speed,  storage 
accuracy,  retrieval  speed,  and  retrieval  accuracy. 

Man's  own  memory  is  often  called  upon  to  provide  a 
substantial  proportion  of  system  memory.  Coal  in- 
formation, function  algorithms,  and  non-stimulus 
data  are  likely  to  be  associated  with  man's  long- 
term memory;  stimulus  data,  with  man's  iconic  and 
short-term  memories  (see  Chapter  5).  Man's  memory 
has  an  astonishingly  great  capacity s but  most  of  the 
stored  information  is  totally  irrelevant  to  functions 
he  is  likely  to  perform  in  surveillance  systems;  its 
storage  speed  and  accuracy  are  highly  dependent  upon 
both  the  man  and  the  learning  situation;  its  infor- 
mation retrieval  speed  can  be  annoyingly  slow;  and 
its  retrieval  accuracy  is  certainly  unreliable 
relative  to  hardware  memory  systems.  Nevertheless, 
man's  own  memory,  when  acting  in  support  of  his  more 
sophisticated  behaviors,  such  as  hypothesis  formula- 
tion or  pattern  recognition,  is  an  invaluable  resource 
which  is  difficult  or  impossible  to  duplicate  with 
engineered  mechanisms. 

Allocation  with  respect  to  the  memory  function  is 
discussed  in  terms  of  four  tasks:  (1)  goal  infor- 

mation, (2)  function  algorithms,  (3)  stimulus  data, 
and  (4)  non-stimulus  data. 

1.1  Goal  information OIR:  5 

The  memory  function  must  provide  storage  of  goal 
information  so  that  the  executive  control  func- 
tion (see  below)  can  attempt  to  bring  about  sys- 
tem goa 1 -di  rected  behavior.  This  goal  information 
should  reflect  system  strategic,  tactical,  and 
technical  employment  considerations,  with  specific 
information  relating  the  importance  and  probability 
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of  occurrence  of  various  targets  of  interest  to 
pertinent  system  capabilities  and  limitations. 

1.2  Function  algorithms OIR:  5 

All  system  functions  we  describe  must  in  some 
sense  be  embodied  in  system  memory.  We  earlier 
noted  that  hard  wiring  constitutes  a form  of 
memory.  Another  example  is  when  a function  is 
executed  entirely  by  a computer.  rn  that  case, 
the  function  algorithm  can  be  precisely  identified 
(consisting  of  a string  of  machine  instructions) 
and  its  location  in  system  memory  can  be  precisely 
identified  physically.  Another,  very  important, 
example  is  when  a function  is  allocated  entirely 
to  man;  in  that  case  neither  the  function  algo- 
rithm nor  its  physic. a 1 location  can  be  precisely 
defined.  Yet  the  importance  of  man's  memory  is 
indisputable,  since  his  behavior,  which  is  crucial 
to  system  functioning,  depends  upon  it. 

1.3  Stimulus  data  . OIR:  2 

In  s u rvc i 1 1 an c e systems,  it  is  often  necessary 
to  acquire,  store,  and  display  large  amounts  of 
stimulus  data. 1 

A common  storage  medium  for  stimulus  data  has 
been  "hard-copy"  displays.  This  medium  has 
well  served  surveillance  systems,  but  it  has 
limitations.  One  is  the  physical  space  required. 
One  alternative  is  to  store  many  millions  of  bits 
of  information  in  such  a manner  that  they  may  be 
retrieved  for  display  in  a timely  manner. 

Another,  generally  unacceptable,  alternative  is 
not  to  store  the  stimulus  data  but  to  rely  in- 
stead upon  real-time  system  functioning  not  to 
overlook  anything  of  importance.  This  is  very 
risky.  Reliance  upon  the  real-time  functioning 
to  select  only  a portion  of  the  stimulus  data 
for  storage  sometimes  leads  to  serious  loss  of 
in  format  ion . Thusj  in  some  surveillance  systems, 
the  stimulus  data  generate  a substantial  memory 
requirement  that  should  be  allocated  to  machines 
if  hard-copy  storage  is  not  feasible. 


*We  use  the  term  "stimulus"  rather  than  signal  or  signal- 
plus-noise  or  signal-plus-noise-plus-clutter  because  the 
definitions  of  signal,  noise,  and  clutter  can  change  dynam- 
ically as  the  system  is  in  operation,  but  these  things  all 
remain  stimuli. 
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1.4  Non-ctirnuluc  data  ........  Oik:  5 

Much  non - s t i mu  1 us  data  is  necessary  for  a sur- 
veillance system  to  operate.  Most  data  are 
technical  intelligence  about  targets  of  interest 
(and,  importantly,  targets  not  of  interest). 

In  early  systems,  a com non  storage  medium  for 
such  data  was  the  printed  page,  in  the  form  nf 
technical  publications.  liven  i n the  latest  sys- 
tems, the  printed  page  remains  important,  not 
only  for  technical  intelligence,  but  also  for 
information  regarding  the  knob  and  dial  opera- 
tion of  hardware.  Printed  operating  data 
gradually  transfer  (with  varying  degrees  of 
fidelity  and  comprehensiveness)  to  the  operator's 
own  memory  through  learning.  llis  memory  consti- 
tutes a second  storage  me din m for  these  data 
and,  when  a complex  function  sulIi  as  classifica- 
tion is  attempted  by  computer,  these  data  will 
be  found  in  a third  medium  of  storage,  the 
computer  memory.  There  (and  in  man,  since  we 
intend  to  model  him  in  dec i s i on - 1 heore t i c terms), 
these  data  are  likely  to  be  represented  in  seine 
combination  of  a priori  probabilities,  likelihood 
functions,  risk  functions,  and  decision  rules. 

Some  where  among  the  non-stimulus  data  should  be 
recorded  the  sum  total  of  the  system's  past 
"experience,"  so  that  the  system  (i.e.,  the 
executive  control  function)  can  deal  with  the 
present  through  historical  perspective  of  the 
past . 

2.  Executive  Control.  In  complex  systems,  we  believe 
the  executive  control  function  necessary  to  bring 
about  integrated,  goa 1 - d i re c ted  system  behavior.  This 
function,  like  others,  may  be  executed  either  by  man 
or  machine.  Most  commonly  it  has  been  within  man's 
province.  Some  of  the  crucial  factors  relating  to 
the  attainment  of  appropriate  goal -directed  behavior 
include  (1)  tiic  ability  to  define  system  goals,  (2) 
the  ability  to  implement  procedures  to  attain  system 
goals,  and  (3)  the  ability  of  the  executive  faithfully 
to  execute  the  procedures. 

Executive  control  requires  interpretation  of  infor- 
mation in  memory  concerning  goals  and  past  situations, 
comprehension  of  the  present  situation,  hypothesis 
generation,  prediction,  effecting  appropriate  con- 
trol responses,  hypothesis  testing,  and  subsequently 
appropriate  iteration.  The  above  are  highly  cognitive 
activities  which  man  performs  with  varying  degrees  of 
effectiveness.  Man's  variability  is  one  of  his 
weaknesses  as  an  executive.  On  the  other  hand,  such 
activities  are  very  difficult  to  automate  success- 
f u 1 1 y . 
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Executive  control  must  exert  its  influence  throughout 
a very  complex  range  of  activities,  from  those  requir 
ing  wide  focus  upon  the  most  molar  aspects  of  system 
functioning  to  those  requiring  narrow  focus  upon 
molecular  detail.  However,  there  arc-  certain  tasks 
characteristic  of  executive  control  at  all  of  these 
levels,  and  wc  will  discuss  function  allocation  with 
respect  to  each  of  them  in  turn:  (1)  comprehension 

of  the  current  situation,  (2)  interpretation  of 
goals,  (3)  interpretation  of  past  situations,  (4) 
hypothesis  generation  and  prediction,  (5j  effecting 
appropriate  control  responses,  and  (6)  hypothesis 
testing  and  iteration. 

2.1  Comprehension  of  the  current 

situation OIK:  5 

The  executive  function  makes  its  control  re- 
sponses based  upon  its  comprehension  of  the 
current  situation,  in  an  attempt  .o  make  that 
comprehension  more  accurate.  The  executive  con- 
trol function  is  adaptive.  Through  iteration, 
it  attempts  to  converge  upon  a true  perspective, 
which  is  the  fundamental  goal  of  any  surveillance 
system.  Accurate  comprehension  of  details  of  the 
situation  at  the  outset  hastens  convergence  and 
reduces  the  probability  that  divergent  behavior 
may  evolve. 

2.2  Interpretation  of  goals OIR:  5 

The  executive  control  function  interprets  the 
goal  information  stored  in  memory  in  the  light 
of  its  perspective  upon  the  current  situation 
in  order  to  bring  about  proper  allocation  of 
system  resources. 

2.3  Interpretation  of  past  situations  OIR:  5 

The  executive  control  function  should  interpret 
the  record  of  past  situations  for  i t s • re  1 at i on- 
ships  to  the  current  situation.  This  activity 
represents  an  "experience  factor"  which  may  have 
considerable  influence  upon  the  control  responses 
made  in  the  current  situation. 

2.4  Hypothesis  generation  and  predic- 

iion OIR:  5 

Based  upon  the  executive's  comprehension  of  the 
current  situation  and  its  interpretation  of  sys- 
tem goals  and  past  situations,  it  will  (ideally) 


form  hypotheses  and  make  predictions.  For 
example,  the  executive  may  determine  that  a tar- 
get is  of  special  interest  and  worthy  of  greater 
attention;  more,  that  if  the  target  is  of  a 
certain  type,  there  should  be  present  a particu- 
lar signal  feature  not  yet  detected. 

2.5  Effecting  appropriate  control 

response  ft 01 R:  5 

Based  upon  predictions  from  the  preceding  task, 
the  executive  should  effect  appropriate  control 
responses.  These  may  be  preliminary,  intermediate, 
or  final  output  responses.  To  continue  our  ex- 
ample (see  2.4),  if  the  executive  predicts  that 
a particular  signal  feature  should  be  present, 
it  should  focus  system  attention  to  heighten 
search  for  that  feature. 

2.6  Hypothesis  testing  and  iteration.  OIR:  5 

After  effecting  control  responses,  the  executive 
tests  its  hypotheses  and  predictions  upon  the 
stimulus  set.  Let  us  continue  our  example:  A 

predicted  target:  signal  is  sought.  If  found,  it 
strengthens  its  associated  hypothesis;  if  not, 
the  hypothesis  is  weakened  or  rejected.  After 
such  a test,  executive  comprehensi on  of  the  cur- 
rent situation  refines  itself.  Then  executive 
control  iterates  its  tasks,  leading  to  continued 
refinement  and,  hopefully,  convergence  on  a 
solut ion . 

5.  Attention  Selection.  The  objective  of  this  func- 
tion is  to  receive  all  necessary  stimuli  to  attain 
system  goals.  This  means  maintaining  surveillance 
of  a sufficient  area  of  stimulus  space  with  suffi- 
cient resolution.  The  area  to  be  searched  is  multi- 
dimensional, including  (1)  the  three  dimensions  of 
lluclidean  space;  (2)  time;  and  (3)  various  signal- 
related  dimensions,  such  as  intensity,  frequency, 
polarization,  and  so  on.  The  crucial  limitation  in 
attention  selection  is  that  the  quotient  of  the 
multidimensional  stimulus  area  to  be  searched  divided 
by  the  resolution  with  which  that  area  is  searched 
has  an  upper  bound.  Since  information  processing 
rate  has  cost  and  technical  constraints,  a trade- off 
is  forced  between  the  area  of  stimulus  space  surveyed 
and  the  resolution  possible  withi..  that  area.  The 
problems  involved  in  making  this  trade-off  are 
substantial . 


15 


(Man  has  his  own,  very  interesting,  attention- 
selection  mechaui  sin , which  is  discussed  at  length 
in  Chapter  3 . ) 

The  attention  selection  function  is  considered  in 
terns  of  four  tasks:  (1)  scarcli  initiation,  (2) 

sensor  placement,  (3)  spatial  coverage  and  spatial 
resolution  selection,  and  (4)  selection  of  other 
parameters . 

3.1  Search  initiation 01 R:  6 

TIi  i s tas)  has  its  most  obvious  meaning  in  connec- 
tion with  certain  systems  that  do  not  ordinarily 
conduct  a continuous  surveillance  operation,  such 
as  certain  airborne  acoustic  ocean  surveillance 
systems;  their  search  must  be  initiated  by  a 
communication  from  the  proper  authority.  This 
communication  must  define  with  some  degree  of 
ccri aintv  the  area  which  is  to  be  searched, 
which  constitutes  the  first  step  in  attention 
selection.  However,  even  in  systems  which  con- 
duct continuous  surveillance  operations,  search 
initiation  still  has  meaning,  for  it  may  refer 
to  an  intensification  of  search  in  some  portion 
of  the  area  under  continuous  surveillance;  we 
shall  refer  to  this  as  "alerted  search."  Alerted 
searching  may  be  brought  about  through  communi- 
cation of  information  from  outside  the  system, 
or  it  may  be  initiated  by  the  system  executive 
function  as  part  of  its  hypothesis  formulation, 
prediction,  and  testing  iteration. 

3.2  Sensor  placement OIR:  6 

For  systems  whose  sensors  are  fixed,  this  task 
hopefully  has  only  to  be  considered  once.  How- 
ever, for  mobile  systems,  the  sensor  placement 
task  constitutes  the  next  step  in  attention 
selection  following  search  initiation,  and 
obviously  defines  to  some  extent  the  spatial 
coverage  of  the  system. 

3.3  Spatial  coverage  and  spatial 

resolution  selection OIR:  5 

In  some  systems,  spatial  coverage  and  spatial 
resolution  arc  not  variable,  but  are  fixed  at 
the  time  of  installation  (granted,  operators  may 
not  give  equal  attention  to  all  spatial  areas 
the  system  senses).  However,  oilier  systems  do 
provide  flexibility  in  these  respects.  Variable 
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coverage  and  resolution  are  particularly  impor- 
tant when  there  is  limited  data  processing 
capacity  and  it  is  necessary  simultaneously  to 
cover  all  potentially  accessible  spatial  areas. 

3.4  Selection  of  other  parameters  . . OIK:  5 

Host  large  systems  have  a large  number  of  mode 
options  and  control  settings  which  must  be 
properly  selected  to  focus  the  system's  atten- 
tion and  permit  optimum  performance.  When  these 
selections  are  made  by  an  operator,  it  frequently 
poses  a significant  challenge  to  him  to  do  so 
effectively;  when  the  selections  are  automated, 
it  often  poses  a significant  challenge  to  the 
designer  to  provide  attenuate  flexibility  in  his 
mode-selection  algorithms  to  responsively  main- 
tain optimality  in  the  face  of  changing  environ- 
mental and  operational  conditions. 

4.  Stimulus  trancmic aion . The  goals  of  this  function 
arc  to  convey  stimuli  to  the  system  sensors  and  then  to 
other  system  elements  as  necessary.  A crucial  factor 
affecting  stimulus  transmission  is  the  environmental 
channel.  Another  is  that  channel  bandwidths  have  cost 
and  technical  limits  which  force  trade-offs  among  types 
of  information  transmitted,  encoding/der.oding  methods, 
and  transmission  rates. 

One  of  the  most  crucial  linl;s  in  man-machine  interaction 
is  at  the  interface  where  stimulus  transmission  occurs 
from  machine  to  man.  This  is  particularly  true  when  the 
complex  stimuli  of  a surveillance  system  are  presented 
visually  to  an  operator  who  must  perform,  or  augment  the 
machine  in  performing,  detection,  feature  extraction, 
and  classification.  Display  hardwarc/sof twarc  technology 
is  in  a relatively  advanced  state  compared  to  our  knowl- 
edge of  the  optimal  formatting  of  complex  information  for 
presentation  to  the  operator.  Much  more  experimental  work 
is  needed  to  add  to  this  knowledge,  particularly  in  the 
face  of  the  trend  toward  displaying  more  information 
(including  new  kinds  of  information  derived  from  more 
advanced  stimulus  processing)  to  fewer  operators. 

The  stimulus  transmission  function  is  considered  below 
in  terms  of  two  basic  tasks:  (1)  pre-sensor  transmission, 

and  (2)  post-sensor  transmission. 

4.1  Pre-censor  tronnmicsion OIR:  N/A 

The  environmental  medium  which  conveys  stimuli  to 
the  sensor  may  have  substantial  effects  on  those 
stimuli,  so  the  nature  of  the  transmission  medium 
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must  be  well  understood.  Many  of  the  apparent 
vagaries  of  various  transmitting  media  have  given 
way  to  knowledge  and  understanding  in  the  face  of 
research.  Still,  there  is  much  that  remains  to  be 
known,  and  that  which  we  may  discover  holds  the 
potential  for  advanced  stimulus  processing  techniques. 
There  is  little  doubt  that  processing  will  remain 
the  province  of  machines. 

4.2  Post-sensor  transmission  OIR:  1 

Once  the  stimuli  have  been  converted  to  electrical 
signals,  they  must  be  routed  throughout  the  system. 
When  the  system  is  sufficiently  compact  physically 
that  the  signals  may  be  carried  by  wires,  there  is 
not  much  of  a problem.  However,  when  parts  of  the 
surveillance  system  are  so  separated  that  electro- 
magnetic transmission  is  necessary  to  convey  data, 
the  quantity  of  those  data,  the  rate  at  which  they 
are  generated,  and  the  need  for  security  in  their 
transmission  conflict  with  cost  and  technical  con- 
straints upon  channel  bandwidth. 

5.  Stimulus  processing . The  goal  of  this  function  is 
to  process  stimuli  to  support  the  detection  and  feature 
extraction  functions.  This  simple  statement  belies  the 
complexity  and  importance  of  the  function.  The  crucial 
factors  here  involve  the  determination  of  optimal  process- 
ing; the  implementation  of  optimal  processing;  and  the 
provision  of  sufficient  processing  flexibility  to  main- 
tain optimality  with  changing  conditions.  Different 
signals  may  require  different  processing,  but  processing 
rate  is  limited  by  cost  and  technical  constraints,  re- 
quiring multiplexing  trade-offs. 

In  surveillance  systems  a large  amount  of  stimulus 
processing  is  done  electronically.  Electronic  process- 
ing enormously  extends  man's  own  sensor  capabilities. 

Some  of  the  most  significant  performance  improvements  in 
the  evolution  of  surveillance  systems  have  resulted  from 
advancements  in  signal  processing  technology.  However, 
at  some  point  the  processed  stimuli  are  usually  presented 
to  an  operator,  at  which  point  his  own  stimulus  process- 
ing must  come  into  play. 

The  stimulus  processing  function  is  considered  in  terms 
of  two  tasks:  (1)  beam  forming,  and  (2)  further 

processing . 

5.1  Beam  forming OIR:  1 
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To  provide  resolution  in  physical  space,  surveillance 
system  sensors  are  usually  formed  into  arrays.  The 


outputs  of  array  sensors  arc  combined  in  various 
ways  to  provide  directional  sensitivity.  Beam- 
forming, in  addition  to  providing  spatial  resolu- 
tion, also  improves  s i gnal - to- noise  ratio.  In 
general,  the  narrower  the  beam  the  better  the  ratio. 

But  narrow  beams  also  mean  that  more  beams  arc 
necessary  for  the  same  coverage,  and  therefore 
greater  processing  and  display  requirements  for  the 
system.  Recent  advances  include  adaptive  beam 
forming,  which  enables  the  system  to  respond  to 
changing  signal  and  noise  conditions.  Most  aspects 
of  beam  forming  do  not  involve  operator  intervention. 

5.2  Further  processing 01 R:  1 

A number  of  additional  processing  techniques  may  be 
employed  to  enhance  specific  characteristics  of  a 
complex  stimulus.  These  are  almost  always  best 
accomplished  with  machines. 

6.  Detection.  The  goal  of  this  function  is  to  detect 
the  presence  of  stimuli  of  interest  in  the  total  stimulus 
set  with  sufficient  speed  and  accuracy.  This  is  a com- 
plex problem  that  requires  knowledge  of  stimuli  of  inter- 
est, of  the  stimulus  set  exclusive  of  stimuli  of  interest, 
and  awareness  of  man  versus  machine  capab Lliti  es/limitations . 
In  many  existing  systems,  man  performs  all  detection 
tasks.  When  target  signal  characteristics  are  well 
known,  detection  can. usually  be  more  effectively  accom- 
plished by  machine. 

Function  allocation  with  respect  to  detection  is  con- 
sidered in  terms  of  three  tasks:  (1)  signal  detection, 

(2)  transient  detection,  and  (5)  track  detection. 

6.1  Signal  detection  OIR:  2 

Signal  detection  consists  of  vertical  or  azimuthal 
search  for  signal  sources,  by  comparing  the  energy 
received  on  one  bearing  or  elevation  with  that  re- 
ceived on  adjacent  bearings  or  elevations.  Signal 
detection  is  probably  best  modeled  in  a sequential 
decision-making  context  (Birdsall  6 Roberts,  1965). 

In  this  model,  two  decision  thresholds  arc  of 
interest.  If  the  received  energy  (or  some  test 
statistic  derived  therefrom)  exceeds  the  higher  of 
these  thresholds,  it  is  decided  that  a signal 

is  present.  If  the  energy  or  test  statistic  falls 
between  these  two  thresholds,  it  is  decided  to  wait 
for  additional  information.  And  (in  the  classical 
model)  if  the  energy  or  test  statistic  falls  below 
the  lower  of  these  thresholds,  it  is  decided  that 
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no  signal  is  present.  In  many  cases,  the  lower  of 
these  two  thresholds  is  inconsequential,  since  the 
system  or  operator  makes  no  overt  response  which 
would  differentiate  between  a decision  to  continue 
searching  and  a decision  that  no  signal  is  present. 
However,  in  other  circumstances  it  is  indeed  con- 
sequential: Consider  the  case  of  an  alerted  search 

for  a signal  which  must  either  be  confirmed  or 
denied  in  order  for  the  executive-control  function 
to  test  predictions  and  refine  hypotheses. 

6.2  Transient,  detection 01R:  8 

The  dictionary  tells  us  that  a transient  "is  short 
in  duration  and  passes  quickly."  The  question  is, 
compared  to  what?  For  surveillance  systems  there 
are  several  frames  of  reference  within  which  the 
definition  might  differ;  in  all,  however,  transients 
are  "tilings  that  happen  somewhat  more  quickly  than 
most  of  what  is  going  on.'1  Whatever  the  frame  of 
reference,  many  transients  share  two  common  charac- 
teristics: 1)  They  may  represent  important  infor- 

mation, and  2)  they  may  be  difficult  to  detect 
automatically.  Provided  sufficient  signal- to -noise 
ratio,  man's  perceptual  mechanisms  make  many  tran- 
sients easily  detectable  on  a suitable  display. 
Automating  their  recognition  and  characterization 
is  far  from  easy,  and  far  from  successful  implemen- 
tation in  any  automatic  system.  (This  accounts,  we 
believe,  for  the  rather  conspicuous  shortcomings  of 
some  highly  automated  systems  we  have  seen). 

6.3  Track  detection OIR:  3 

Target  motion  parameters  are  extremely  important  in 
surveillance  systems,  both  for  maintaining  contact 
and  to  assist  in  determining  the  tactical  and  stra- 
tegic implications  of  threat  targets.  Track  detec- 
tion, and  target  localization  and  motion  analysis, 
can  be  most  precisely  executed  using  the  outputs 
of  several  sensors  at  geographically  separated 
locations . 

7.  Feature  cxtraction/acsociation.  The  goals  of  this 
function  are  to  extract  relevant  features  from  the  stimuli 
of  interest,  associate  these  features  where  appropriate, 
and  extract  relevant  features  resulting  from  the  associa- 
tion in  an  iterative,  hi  rarchical  process  of  extraction 
and  aggregation,  leading  to  more  and  more  features.  The 
crucial  factors  arc  definition  of  relevant  features  and 
of  appropriate  associations,  and  the  development  of 
feature  extraction  and  association  methodologies.  These 


processes  arc  complicated  by  the  necessity  for  invariance 
to  irrelevant  transformations  or  changes  in  the  stimulus 
set,  and  adaptability  to  relevant  transformations  or 
changes  in  the  stimulus  set. 

For  signals  with  stationary  probability  distributions, 
machines  can  exceed  man  in  reliability,  accuracy,  and 
speed  of  signal  parameter  estimation.  However,  estima- 
tion of  parameters  which  characterize  non- s tat i onary  sig- 
nals is  probably  best  performed  by  man.  Likewise, 
screening  can  often  be  efficiently  performed  by  man, 
since  his  attention  selection  mechanisms  permit  him  to 
ignore  stimuli  that  are  not  of  interest  and  attend  to 
those  that  arc. 

The  functions  associated  with  feature  extraction  and 
association  will  be  discussed  in  relation  to  seven  tasks: 
(1)  signal  parameter  estimation  (extraction  of  lowest- 
ordcr  features),  (2)  screening,  (3)  early  classification 
based  on  especially  distinctive  lower-order  features,  (4) 
association  of  lower-order  features  into  appropriate 
higher-order  features,  (5)  alerted  searching  for  unde- 
tected lower-order  features  based  or.  expected  associations , 

(6)  association  of  higher-order  features  with  targets,  and 

(7)  classification  based  on  especially  distinctive  higher- 
order  features. 

7.1  Signal  parameter  estimation  (extrac- 
tion of  lowest-order  features) . . . .OIR:  6 

In  order  to  support  the  target  localization,  motion 
analysis,  and  classification  functions,  appropriate 
features  must  be  extracted  from  received  signals. 

The  features  of  interest  can  be  considered  random 
variables,  and  therefore  their  characteristics  will 
be  derived  in  a statistical  sense. 

However,  the  probability  distributions  of  the 
variables  of  interest  often  arc  not  stationary; 
typically  they  exhibit  both  time  and  space  depen- 
dencies. Therefore,  each  of  the  extracted  statisti- 
cal parameters  must  have  associated  with  it  the  time 
for  which  it  was  derived  and  an  appropriate  spatial 
specification.  The  more  radically  the  probability 
distributions  of  the  signals  of  interest  depart  from 
s t a t i onar it y , the  more  they  approach  what  we  charac- 
terized earlier  as  "transients";  in  that  case  they 
present  a situation  in  which  non-stationary  behavior 
may  represent  very  important  information  that  may 
be  difficult  to  automatically  characterize.  In 
contrast,  the  characterization  of  random  variables 
that  have  stationary  probability  distributions,  by 
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means  of  automated  feature  extraction  techniques, 
is  a problem  that  may  he  approached  with  some  con- 
fidence. It  has  been  shown  that  a machine  can 
exceed  man  in  the  precision  of  sucli  extraction. 

However,  the  problem  of  adequately  characterizing 
radically  nou-stationary  processes  automatically  can 
be  approached  with  much  less  confidence.  Substantial 
efforts  and  some  progress  have  been  made  in  this 
area,  but  we  do  not  believe  that  it  has  been  demon- 
strated that  machines  can  presently  equal  the  capa- 
bilities of  man  in  adequately  extracting, 
characterizing,  and  utilizing  these  kinds  of 
features. 

7.2  Screening OIR:  6 

Based  upon  signal  parameter  estimation,  it  can  usually 
be  concluded  that  certain  signals  do  not  require 
further  immediate  attention.  Examples  of  such  sig- 
nals are  equipment-  or  platform-generated  artifacts 
or  artifacts  associated  with  the  transmission  medium, 
signals  whicli  have  already  been  accounted  for,  and 
signals  that  have  not  yet  been  accounted  for  but 
which  are  not  believed  associated  with  targets  re- 
quiring immediate  attention.2 

7.3  Early  classification  baaed  on  espe- 

cially distinctive  characteristic 
loner-order  features  OIR:  2 

Certain  targets  may  produce  especially  distinctive 
signals  which  lend  themselves  to  automatic  detection 
and  consequent  early  classification.  However,  such 
classifications  are  not  very  often  certain  because 
of  probable  overlap  of  attributes  of  targets  from 
different  classes. 


Association  of  lower-order  features 

into  appropriate  higher-order  features  OIR:  8 


The  basic  parameters  of  a signal  can  often  be  inter- 
related in  such  a way  as  to  draw  inferences  about 
the  physical  features  or  behavior  of  a target. 
These  in  turn  lead  to  probability  statements  about 

2 

Certain  unchanging  artifacts,  characteristic  of  a specific 
system,  may  actually  be  screened  out  during  stimulus  processing, 
reducing  the  processing  load  during  the  detection  and  feature 
extraction  functions.  However,  there  remains  a significant 
subset  of  the  total  stimulus  set  which  is  dynamically  changing 
and  therefore  cannot  be  eliminated  during  early  processing. 
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the  target's  nature.  This  is  a classic  problem  in 
pattern  recognition,  the  solution  to  which  is  very 
difficult  to  automate,  particularly  when  the  signal 
space  is  occupied  by  many  targets  at  a given  time. 

At  this  stage  of  development,  man's  role  in  these 
processes  remains  critical. 

7.5  Alerted  searching  for  undetected 

lower- order  features  OIR:  2 

Once  tentative  identification  of  a signal  pattern 
is  made,  one  may  predict  the  appearance  of  other 
pattern  elements  even  if  they  are  not  initially 
detected.  Knowing  the  characteristics  of  these 
undetected  pattern  components  permits  alerted  search 
for  the  remainder  of  the  pattern,  likely  by  refocus- 
ing system  attention. 

7.6  Association  of  higher-order  features 

with  targets OIR:  8 

This  activity  consists  of  associating  together  all 
hxgher-order  features  that  appropriately  belong  to 
a particular  target  and  carefully  excluding  features 
that  belong  to  other  targets,  so  that  proper  clas- 
sification of  the  target  can  take  place.  The  task 
is  made  difficult  by  the  simultaneous  presence  of 
many  signal  sources  and  overlapping  target  charac- 
teristics. Again,  the  pattern  recognition  capabili- 
ties of  man  appear  essential  for  success.  It  is 
significant  to  note  that  many  potential  clues  to 
target  association  have  been  omitted  in  the  various 
attempts  to  automate  this  task.  It  is  therefore 
not  surprising  that  no  attempt  at  automated  target 
association  that  we  know  of  will  function  adequately 
in  a stimulus-rich  environment. 

7.7  Early  classification  based  on  espe- 

cially distinctive  characteristic 
higher-order  features OIR:  2 

Somewhat  infrequently,  a distinctive  combination  of 
target  features,  derived  from  a combination  of  lower- 
order  signal  features,  will  permit  classification  of 
the  target  with  high  confidence.  The  emphasis  is  on 
especially  distinctive  , which  permits  automation  to 
be  more  successful  in  this  case  than  in  others  where 
the  pattern  to  be  recognized  is  less  obvious. 

8.  Feature  space  transformation.  The  goals  of  feature 
space  transformation  include  reducing  the  dimensionality 
of  feature  space  to  reduce  system  information  processing 
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load;  increasing  discriminabi 1 ity  so  that  classification 
can  be  done  in  the  feature  space  which  yields  the  best 
results;  and,  sometimes,  coordinate  transformation,  for 
example,  to  describe  target  location  and  motion  in  an 
appropriate  coordinate  system.  The  crucial  factors 
involve  discovering  appropriate  transformations,  particu- 
larly since  dimensionality  reduction  and  d iscriminab i 1 i ty 
enhancement  are  often  inversely  related. 

With  respect  to  man-machine  function  allocation,  trans- 
formation of  features  to  reduce  dimensionality  and  maxi- 
mize discriminabil ity  is  something  of  a gray  area.  If 
man  is  acting  as  the  principal  feature  extractor/ associa- 
tor,  he  will  perform  some  feature  space  transformations 
on  his  own.  To  the  extent  that  some  of  the  following 
tasks  are  automated,  however,  they  may  involve  further 
feature  space  transformations  by  computer  algorithm. 

Feature  space  transformation  will  be  described  in  terms 
of  three  tasks:  (1)  combination  of  features  to  reduce 

dimensionality,  (2)  combination  of  features  to  maximize 
discriminabil ity , and  (3)  coordinate  transformation. 

8.1  Combination  of  features  to  reduce 

dimensionality OIR:  5 

As  might  be  inferred  from  our  discussion  of  the  fea- 
ture extraction/association  function,  an  extensive 
analysis  of  surveillance  system  stimuli  can  lead  to 
a quite  large  number  of  features  of  various  types 
and  diagnostic  potential.  The  independent  processing 
of  all  these  features  during  subsequent  downstream 
functions  may  burden  the  system  with  an  undesirable 
or  even  impossible  load,  and  it  is  therefore  almost 
always  desirable  to  apply  some  transformation  to  the 
feature  space  as  it  is  originally  extracted  so  as  to 
reduce  its  dimensionality.  The  simplest  technique 
for  doing  this,  which  is  absurd,  is  to  omit  features 
randomly  until  the  processing  load  is  reduced  to  a 
tolerable  or  desirable  level.  It  is  easy  to  demon- 
strate that  machines  do  not  do  this,  rather,  that 
programmers  do  not  design  feature  space  transforma- 
tions in  this  way.  It  is  not  so  easy  to  demonstrate 
that  human  operators  do  not  occasionally  do  this. 

In  any  event,  if  omission  is  to  be  the  technique  of 
dimensionality  reduction,  it  is  obviously  more 
sensible  to  be  selective  in  picking  what  is  to  be 
omitted.  In  system  design  stages,  this  is  sometimes 
accomplished  by  omitting  the  features  that  are  most 
difficult  to  extract,  leading  to  a calculated  failure 
to  extract  them  at  all.  While  this  may  be  justifi- 
able on  the  grounds  of  cost  or  technical  constraints, 
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many  of  the  shortcomings  of  automation  in  surveil- 
lance systems  can  be  traced  to  the  omission  of  such 
features . 

Another,  more  desirable  way  to  reduce  dimensionality 
is  to  use  a linear  combination  of  features,  wherein 
the  coefficients  are  allowed  to  take  on  appropriate 
values  according  to  the  diagnostic  potential  of  the 
features.  One  ~z  not  limited  to  a linear  combination 
of  features.  The  combination  may  be  more  complex. 

It  is  easy  to  imagine,  for  example,  that  feature 
space  transformations  that  occur  within  human  opera- 
tors may  be  quite  difficult  to  define  in  simple 
mathematical  terms.  Regardless  of  the  mechanism, 
the  gOu  of  dimensionality  reduction  is  almost  always 
tempered  by  the  concern  to  conserve  information  and 
maximize  discriminability . 

8.2  Combination  of  features  to  maximize 

discriminability  OIR:  5 

The  appropriate  combination  of  features  to  maximize 
d sc  iminability  is  not  easy  to  specify.  Coupled 
with  the  need  to  reduce  feature  space  dimensionality, 
arriving  at  an  appropriate  means  to  combine  features 
becomes  a problem  of  magnitude  second  only  to  ex- 
tracting features  in  the  first  place.  If  we  con- 
sider feature  space  transformations  that  can  be 
executed  in  computers  to  reduce  dimensionality  and 
maximize  or  minimize  the  loss  of  discriminability, 
we  can  define  what  is  going  on,  if  not  demonstrate 
that  what  is  going  on  is  optimal. 

For  example,  in  a multivariate  discriminant  analysis 
approach,  the  weights  or  coefficients  to  be  used  in 
the  linear  combination  of  features  for  the  construc- 
tion of  each  dimension  in  a new  feature  space  are 
derived  from  the  components  of  the  eigenvectors 
resulting  from  the  solution  of  an  eigenvalue  problem 
so  structured  that  between-c 1 as s distances  are  maxi- 
mized with  respect  to  within-class  dispersions. 
Naturally,  the  example  target  classes  which  serve 
as  a data  base  for  the  eigenvalue  problem,  or  for 
any  other  approach  to  feature  space  transformation, 
must  consist  of  a very  substantial  and  representative 
library  of  actual  target  signals  of  various  classes 
from  which  the  features  to  be  transformed  are  to  be 
extracted.  To  the  extent  that  various  target  class 
dispersions  in  the  original  feature  space  conform 
to  the  assumptions  of  the  particular  technique  being 
used  (for  example,  the  assumption  of  multivariate 
normal  distributions,  the  equality  of  dispersions 
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among  all  target  classes,  or  whatever),  multivariate 
statistical  techniques  may  provide  optimum  solutions 
to  the  weighting  of  the  various  features  for  the  pur- 
pose of  transformation  to  a space  where  dimension- 
ality may  be  reduced  while  minimizing  loss  of 
discriminability  (e.g.,  by  omitting  dimensions  with 
the  smallest  associated  eigenvalues). 

8.3  Coordinate  transformation OIR:  1 

Certain  features  extracted  in  the  course  of  surveil- 
lance system  functioning  need  to  be  transformed  to 
more  convenient  or  appropriate  spaces  in  a perfectly 
obvious  and  well-defined  mathematical  manner;  it 
seems  clear  to  us  that  such  transformations  should 
be  done  by  machine. 

9.  Target  localization.  The  goal  of  target  localization 
is  to  determine  target  position  parameters  in  physical 
space  with  sufficient  speed  and  accuracy.  Crucial  factors 
include  system  resolution  in  physical  space  and  the  ability 
to  track  moving  targets.  Target  localization  requires 
precise  estimation  of  certain  parameters,  difficult  for 
man  to  accomplish,  and  a large  variety  of  mathematical 
computations.  This  function  therefore  is  particularly 
suited  to  automation. 

« 

The  target  localization  function  is  described  in  terms  of 
three  tasks:  (1)  single-sensor  fixing,  (2)  multiple- 

sensor fixing,  and  (3)  tracking. 

9.1  Single-sensor  fixing  OIR:  2 

The  simplest  case  of  single-sensor  fixing  involves 
target  localization  using  an  omnidirectional  sensing 
device.  Omnidirectional  sensors  do  not  furnish 
azimuthal  information.  However,  if  such  a sensor 
has  limited  range  capability,  it  can  provide  a useful 
circular  area  of  probability  within  which  the  tar- 
get might  lie.  In  systems  which  provide  beam-forming, 
target  azimuth  may  be  estimated.  Range  may  be  pro- 
jected to  some  degree  if  target  bearing  rate 
and  speed  are  known  or  can  be  inferred  through 
.intelligence  or  other  sources  of  information. 

With  a single  passive  sensor,  an  estimate  of  target 
range  is  difficult  to  define  with  adequate  precision. 
An  active  system,  of  course,  can  provide  both  azi- 
muth and  range. 

9.2  Multiple-sensor  fixing  OIR:  2 

Cross-fixing  involves  bearing/range  determinations 
on  the  same  target  from  two  or  more  sensors  which  are 


at  different  geographic  locations.  A multiple-sensor 
fix  is  generally  more  accurate  than  a single-sensor 
fix.  An  important  consideration  in  cross-fixing  is 
communication  between  machines  or  operators  attending 
the  various  sensors.  The  outputs  of  several  sensors 
may  be  brought  together  at  a single  location,  in 
which  case  this  communication  may  be  done  directly. 

If  the  outputs  terminate  at  different  geographic 
locations,  complications  in  communication  are  intro- 
duced. The  first  requirement  in  cross-fixing  is  to 
ensure  that  the  various  sensors  are  looking  at  the 
same  target.  This  relates  to  the  task  of  associating 
features  with  targets,  which  can  be  facilitated  by 
the  correlation  of  information  among  sensors,  either 
at  the  stimulus  level  or  at  some  higher  system 
output  level.  We  will  discuss  this  further  when  we 
take  up  the  communication  function. 


9.3  Tracking 


OIR:  2 


The  term  "tracking"  denotes  the  problem  of  success- 
fully maintaining  target  localization  over  a period 
of  time,  which  is  significant,  and  which  can  be 
aided  by  the  approaches  we  cite  to  target  localiza- 
tion and  by  the  various  methods  of  determining  tar- 
get motion  parameters  discussed  beneath  that  function. 
Successful  tracking  provides  an  input  to  target 
motion  analysis,  and  successful  target  motion  analy- 
sis, which  has  inputs  other  than  position-versus - 
time  analysis,  will  aid  in  successful  tracking. 

10.  Target  motion  analysis . The  goal  of  target  motion 
analysis  is  to  determine  target  motion  parameters  with 
sufficient  speed  and  accuracy.  The  crucial  factors 
include  system  resolution  in  physical  space  and  in  other 
target  motion-related  feature  dimensions  (e.g.,  doppler 
shift)  and  the  ability  to  track  moving  targets.  Target 
localization,  target  motion  analysis,  and  the  ability  to 
track  moving  targets  are  all  closely  related. 


However,  we  feel  that  localization  and  motion  analysis 
are  sufficiently  different  that  each  deserves  a place  on 
our  list  of  functions.  Tracking  is  accomplished  de  facto, 
when  localization  is  successfully  maintained  over  a 
period  of  time.  We  have  listed  "ability  to  track"  as  a 
crucial  factor  in  localization  and  motion  analysis  in  the 
sense  in  which  it  reflects  a system's  tolerance  to  chang- 
ing parameters. 


We  would  expect  a human  operator  to  review  the  outputs 
of  automated  localization,  tracking,  and  motion  analysis 
algorithms,  because  the  experienced  operator  develops 
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insights  regarding  system  functioning  and  probable  target 
behaviors  that  can  permit  detection  of  errors  in  the 
automatic  execution  of  these  tasks. 

Target  motion  analysis  is  considered  in  terms  of  three 
tasks:  (1)  inference  of  motion  based  on  extracted  stimu- 

lus features,  (2)  position-versus-t ime  analysis,  and 
(3)  track  aiding. 

10.1  Inference  of  motion  based  on 

extracted  stimulus  features OIR:  2 

Some  stimulus  features  relate  directly, 
or  indirectly,  to  target  speed.  Analysis  of  these 
features,  together  with  knowledge  of  target  class, 
makes  possible  inferences  about  target  motion. 

10.2  Position-versus-time  analysis . . . . OIR:  2 

Successive  localizations  of  a target  over  a suffi- 
cient period  of  time  serve  to  indicate  target  motion 
parameters.  Currently,  position-versus-time  analy- 
sis is  conducted  in  some  instances  manually.  How- 
ever, we  believe  that  this  task,  and  most  os  the 
others  listed  under  target  localization  and  target 
motion  analysis,  are  obvious  candidates  for  auto- 
mation, because  solutions  can  be  achieved  with 
greater  speed  and  accuracy.  The  use  of  automation 
is  growing,  but  we  feel  that  there  is  room  for 
broader  application. 

10.3  Track  aiding  OIR:  2 

As  mentioned  above,  the  results  of  target  motion 
analysis  are  useful  in  target  localization.  First, 
knowing  target  motion  parameters  permits  prediction 
of  a track,  assuming  the  parameters  do  not  change. 
Second,  the  nature  of  motion  parameter  changes  may 
sometimes  be  inferred  from  changes  in  lower-order 
features  of  the  signal,  before  the  target  departs 
noticeably  from  its  predicted  track;  this  may  enable 
more  timely  modification  of  the  track. 

11.  Classification.  The  goals  of  the  classification 
function  are  to  determine  what  a target  is,  what  it  is 
doing,  and  why,  with  sufficient  speed  and  accuracy. 

Much  attention  has  been  focused  upon  this  function,  and 
some  of  the  crucial  factors  relating  to  it,  put  into 
decision- theoret ic  terms,  are:  knowledge  of  target  class 

likelihood  functions  in  appropriate  feature  spaces;  knowl- 
edge of  a.  priori  probabilities;  knowledge  of  decision 
risk  functions;  knowledge  of  appropriate  decision  rules; 
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and  the  capability  to  execute  logical  decision  processes. 
The  execution  of  the  classification  function  typically 
places  heavy  demands  upon  the  memory  function. 

Classification  is  a decision-making  function.  There  is 
a large  and  rather  bewildering  body  of  literature  re- 
garding the  decision-making  behavior  of  human  beings  (see 
Rapoport  and  Wallsten  [1972]).  Here  we  shall  only  attempt 
to  paint  with  a broad  brush  the  outlines  of  human  decision 
behavior  as  we  feel  they  apply  to  classification  in  sur- 
veillance systems. 

Classification  depends  upon  adequate  execution  of  the 
feature  extraction/association  function.  Man  is  generally 
superior  to  machine  in  performing  most  aspects  of  that 
function.  However,  to  successfully  employ  the  extracted 
features,  the  classification  function  calls  for  appropri- 
ate and  sometimes  complex  aggregations  of  these  features 
and  requires  comparison  of  these  aggregations  to  very 
large  amounts  of  information  held  in  some  form  by  system 
memory.  Now,  man  as  a logical  processor  is  known  to  be 
somewhat  faulty;  man  as  an  information  storage  device  is 
also  less  than  perfect.  These  characteristics  of  man 
almost  certainly  impair  his  performance  of  the  classifi- 
cation function  to  some  degree. 

We  suggest  that  there  are  two  approaches  which,  taken 
either  independently,  or  preferably,  together,  should 
improve  system  functioning.  First  of  all,  we  have  no 
doubt  whatever  that  man's  classification  performance 
could  be  substantially  improved  by  the  appropriate  selec- 
tion, training,  and  motivation  of  surveillance  system 
operators.  Second,  wc  feel  that  the  logical  processes 
involved  in  the  classification  function  are  suitable 
candidates  for  automation.  We  arc  inclined  to  believe 
that  a combination  man-machine  approach  to  the  classifi- 
cation function  could  bring  about  substantial  system 
performance  improvements. 

Two  things  are  worthy  of  mention  at  this  point  regarding 
our  discussion  of  classification.  First,  .the  reader  will 
notice  a flavor  of  decision  theory  in  our  organization 
and  description  of  various  classification  tasks.  We 
think  decision  theory  provides  an  appropriate  framework 
within  which  to  structure  a description  of  classifica- 
tion, but  we  have  attempted  to  maintain  sufficient 
generality  so  that  our  discussion  may  apply  equally  well 
to  the  tasks  as  they  are  executed  by  men  or  various 
kinds  of  machines,  at  least  in  the  sense  of  a model. 
Second,  not  all  the  following  tasks  will  necessarily  be 


29 


executed  by  a human  operator  or  a machine  for  a given 
classification  problem;  some  of  them  may  rarely  be 
executed,  and  some  of  them  may  not  even  be  within  the 
repertoire  of  behavior  of  a particular  system.  Nonethe- 
less, to  maintain  adequate  generality,  we  feel  it  neces- 
sary to  touch  upon  each  of  the  following  tasks  in  turn. 

The  classification  function  will  be  discussed  in  terms 
of  eleven  tasks:  (1)  determination  of  stimulus  source 

likelihood  estimates,  (2)  alerted  searching  for  undetected 
signals  typically  related  to  inferred  stimulus  sources, 

(3)  determination  of  stimulus  source  configuration  like- 
lihood estimates,  (4)  determination  of  operating  behavior 
likelihood  estimates,  (5)  determination  of  target  class 
likelihood  estimates,  (6)  alerted  searching  for  unde- 
tected signals  typically  related  to  inferred  target 
classes,  (7)  determination  of  target  class  a priori 
probabilities,  (8)  determination  of  target  class  a 
posteriori  probabilities,  (9)  determination  of  classifi- 
cation decision  risk  functions,  (10)  determination  of 
an  appropriate  decision  rule,  and  (11)  classification 
decision  making. 

11.1  Determination  of  stimulus  source 

likelihood  estimates OIR:  5 

This  task  may  be  defined  as  estimating  the  likeli- 
hood that  a particular  signal  characteristic  would 
be  produced  by  a particular  physical  source.  It  may 
be  executed  in  a number  of  ways,  for  example,  by 
stimulus  matching  techniques,  by  multivariate  sta- 
tistical analysis,  or.  in  the  case  of  the  expert 
operator,  by  recognition.  In  any  event,  the  task 
will  place  heavy  demands  upon  the  system  memory 
function,  which  in  some  form  or  other  must  store 
the  necessary  technical  intelligence  information, 
in  addition  to  the  task  algorithm. 

11.2  Alerted  searching  for  undetected 
signals  typically  related  to 

inferred  stimulus  sources  OIR:  2 

The  physical  sources  thought  likely  to  be  responsible 
for  certain  stimulus  components  that  have  been  de- 
tected may  be  known  to  commonly  produce  other  com- 
ponents that  have  not  yet  been  detected.  In  that 
case,  an  alerted  search  for  these  undetected  com- 
ponents may  be  initiated. 

11.3  Determination  of  stimulus  source 

configuration  likelihood  estimates . OIR:  4 

This  task  is  similar  to  11.1  but  involves  relation- 
ships between  the  signal  characteristics  and  various 
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characteristics  of  targets  rather  than  a direct 
relationship  with  target  type  itself.  Thus  the 
inference  of  target  type  is  more  remote. 

11.4  Determination  of  operating  behavior 

likelihood  estimates OIR:  7 

By  "operating  behavior,"  we  mean  the  answer  one 
would  hope  to  be  able  to  give  to  the  question,  "What 
has  this  target  been  doing?"  This  is  an  important 
question  to  answer  correctly,  because  that  answer 
can  help  considerably  in  answering  the  key  question, 
"What  is  this  target?"  And,  when  both  of  these 
questions  can  be  answered,  it  sometimes  becomes 
evident  why  the  target  is  behaving  as  it  is.  Expert 
operators  infer  operating  behavior  from  target 
histories  all  the  time,  and  put  those  inferences 
to  crucial  use  in  answering  the  critical  classifi- 
cation questions  of  surveillance  systems.  Pro- 
gramming a computer  to  do  the  same  thing  can  be  a 
formidable  task. 

11.5  Determination  of  target  class 

likelihood  estimates OIR:  3 

This  task  consists  of  estimating  the  likelihood  that 
target  signal  "i"  would  be  produced  by  a target  of 
class  "j"  for  all  appropriate  "j"  and  "j."  The 
target  classes  under  consideration  may  be  very 
broad  categorizations,  such  as  threat  and  non- 
threat,  or  they  may  represent  more  specific  types. 
The  determination  of  target  class  likelihood  esti- 
mates may  be  based  upon  any  or  all  of  the  preceding 
tasks  we  have  discussed  under  classification,  or 
it  may  be  based  upon  some  so-t  of  Gestalt  technique 
which  docs  not  explicitly  involve  the  preceding 
tasks  in  any  obvious  way.  However  it  is  executed, 
the  task  of  determining  target  class  likelihood 
estimates  is  without  doubt  one  of  the  most  important 
in  surveillance  systems,  and  it  is  the  task  in  which 
all  the  other  stimulus-related  functions  and  tasks 
culminate.  It  also  requires  very  substantial  sup- 
port from  the  system  memory  function  to  permit  the 
association  of  target  signals  of  unknown  origin  to 
the  known  characteristics  of  targets  of  various 
classes  . 

11.6  Alerted  searching  for  undetected 
signals  typically  related  to 

inferred  target  classes  OIR:  3 

Once  it  is  inferred  that  a signal  may  be  generated 
by  a certain  class  target,  it  can  be  determined 
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whether  it  contains  all  the  components  typically 
generated  by  that  class.  If  it  does  not,  an  alerted 
search  may  then  be  initiated  in  an  attempt  to  detect 
the  missing  components. 

11.7  Determination  of  target  class 

a priori  probabilities OIR:  4 

Target  class  likelihood  estimates  as  we  have  defined 
them  arc  derived  solely  from  sensor  stimulus  data 
(along  with  necessary  supporting  information)  without 
regard  to  the  relative  frequency  of  occurrence  of 
targets  of  various  classes,  or  with  respect  to  the 
prior  probabilities  of  detecting  targets  of  given 
classes  based  on  any  other  considerations.  The 
determination  of  a priori  probabilities  in  our 
structuring  of  the  classification  function  is  re- 
served as  in  independent  task,  which  we  arc  now 
defining.  Estimates  of  a priori  probabilities  for 
target  classes  and  their  behaviors  may  be  based 
upon  information  derived  from  other  sensors  or  other 
sources  of  intelligence.  For  example,  it  may  be 
known  that  a target  of  a given  class  is  "tracking" 
in  such  a manner  that  it  will  probably  be  detected 
by  a given  surveillance  system,  leading  to  a rather 
high  a priori  probability.  The  derivation  of  a 
priori  probabilities  usually  involves  a component 
of  subjective  evaluation  by  some  human  judge,  during 
programming  cud/or  in  real  time. 

11.8  Determination  of  target  class 

a posteriori  probabilities OIR:  2 


In  this  task,  the  target  class  likelihood  estimates 
are  combined  with  the  target  class  a priori  proba- 
bilities (hopefully,  with  Bayes'  rule)  to  derive  a 
posteriori  probabilities  that  a target  of  class  "i" 
is  actually  producing  target  signal  "j,"  for  all 
appropriate  "i"  and  "j."  These  probabilities  will 
be  utilized  in  the  decision-making  tasks  which  fol- 
low, but  their  values  are  of  considerable  indepen- 
dent interest  and  may  be  reported  along  with  the 
results  of  actual  classification  decision-making, 
since  they  serve  as  indicators  of  confidence  re- 
garding the  final  decision  and  possible  alternative 
decisions  . 

11.9  Determination  of  classification 

decision  risk  functions  OIR:  3 

This  task  must  rely  upon  the  executive  and  memory 
functions,  and  appropriate  inputs  thereto,  to  derive 
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estimates  of  the  "risks,"  "costs,"  or  "utilities" 
of  the  various  possible  target  classification  out- 
comes. In  many  instances  these  will  be  estimated 
in  a highly  subjective  manner,  and  sometimes  quite 
inappropriately. 

11.10  Determination  of  an  appropriate 

decision  rule OIR:  2 

An  appropriate  decision  rule  must  be  selected  to 
come  to  a classification  decision.  Various  decision 
rules  might  be  employed,  such  as  a "maximum  likeli- 
hood" rule,  in  which  the  target  will  be  assigned  to 
that  class  which  shows  the  highest  a posteriori 
probability;  or,  in  an  even  more  rudimentary  scheme, 
to  that  class  which  has  the  highest  likelihood 
estimate  thus  ignoring  the  a priori  pr obab i 1 i t i es ; 
or,  rules  which  consider  the  risk  functions,  such  as 
Hayes'  rule,  which  permit  minimization  of  the  expected 
risk  or  cost,  where  the  a priori  probabilities  are 
reasonably  well-known;  or  Minimax  rules,  which  mini- 
mize the  maximum  conditional  risk,  where  the  a priori 
probabilities  arc  less  certain;  or,  a Neyman-Pcarson 
rule,  which  attempts  to  maximize  the  correct  classifi- 
cations while  bolding  the  false  alarms  to  some  preset 
value  which  is  determined  cither  implictly  or  explicitly 
from  cost  and  a priori  probability  considerations.  In 
automated  decision-making  systems,  the  rule  or  rules 
employed  will  obviously  be  uniquely  identifiable;  when 
man  is  the  decision-maker,  it  will  not  be  so  easy  to 
determine  what  rule  or  rules  are  operating. 

11. J]  Classification  decision  making.  . . OIR:  3 

Based  upon  the  foregoing  tasks,  decisions  may  be 
made  as  to  what  a target  is,  what  the  target  is 
doing,  and  why  the  target  is  doing  it.  Depending 
upon  the  nature  of  the  target's  signal  and  the 
quantity  and  quality  of  the  information  and  algo- 
rithms necessary  to  support  the  foregoing  tasks, 
these  decisions  will  be  made  with  varying  degrees 
of  confidence.  The  output  of  this  task  is  crucial. 

It  is  the  raison  d’etre  of  surveillance  systems. 

12.  Communication.  The  goals  of  this  function  are  to 
provide  channels  to  permit  coordinated  surveillance  sys- 
tem operation,  to  convey  surveillance  system  outputs  to 
external  agencies,  and  to  receive  instructions  and  other 
information  from  external  agencies.  The  communication 
function  is  subject  to  constraints  upon  encoding/decoding, 
speed,  accuracy,  reliability,  and  channel  bandwidth. 
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Ideally,  the  development,  formatting,  encoding,  trans- 
mission, decoding,  interpretation,  storage,  and  retrieval 
of  messages  necessary  to  achieve  integrated  system  func- 
tioning must  occur  in  as  timely,  smooth,  and  error-free 
manner  as  possible.  The  communication  function  as  it  is 
often  executed  is  archaic.  Men  are  involved  in  all  sorts 
of  information  handling  tasks  that,  given  the  potential 
of  present-day  technology,  they  should  r.ot  be. 

The  communication  function  can  be  broken  down  in  terms  of 
three  tasks:  (1)  system  coordination,  (2)  system  out- 

put, and  (3)  system  input. 

12.1  System  coordination  OIR:  3 

The  functions  and  tasks  of  detection,  feature  ex- 
traction and  association,  localization,  target 
motion  analysis,  and  classification  require  or  are 
greatly  facilitated  by  the  coordination  of  infor- 
mation from  various  sensors.  Ideally,  the  network 
of  surveillance  systems  of  all  types  should  be  so 
well  integrated  that  they  might  be  regarded  as 
representing  a single  species  of  surveillance  sys- 
tem. Consequently,  cne  of  the  most  important 
portions  of  the  executive  control  function  could 
then  be  devoted  to  the  centralized  control  of  the 
integrated  system.  However,  successful  execution 
of  integrated  control  depends  heavily  upon  the  com- 
munication function. 

12.2  System  output OIR:  3 

Surveillance  systems  exist  to  provide  strategic, 
tactical,  and  technical  intelligence  to  user  agencies. 
The  user  agency  may  or  may  not  be  closely  linked  with 
the  operating  agency.  In  any  event,  the  communica- 
tion function  must  provide  system  output  to  the  user 
agencies  in  an  accurate  and  timely  manner. 

12.3  System  input OIR:  3 

User  agencies  may  make  inputs  into  a surveillance 
system  to  alert  the  system  to  the  possible  presence 
of  targets,  to  request  increased  attention  to  tar- 
gets that  are  present  or  are  likely  to  be  held  by 
the  system,  and  to  provide  intelligence  information 
from  other  sources  that  may  enhance  system  performance. 
Inputs  from  user  agencies  are  obviously  very  important, 
since  surveillance  systems  exist  to  serve  those 
agencies.  Both  system  input  and  output  appear  to 
be  candidates  for  consider.?1  1e  automation. 


13.  Learning.  The  effects  of  learning  should  be  two- 
fold. First,  they  should  result  in  modification  of  sys- 
tem behavior  in  such  a way  as  to  improve  performance. 
Second,  they  should  reveal  new  or  changed  target  charac- 
teristics or  targets  to  improve  strategic,  tactical,  and 
technical  intelligence.  These  results  are  closely  related. 
Functionally,  systems  learning  is  embodied  in  modifications 
to  the  system  memory  function.  This  applies  to  both  man 
and  machine,  and  can  come  about  internally,  in  adaptive 
systems,  or  through  reprogramming  by  an  external  agency. 

The  learning  function  is  broken  down  into  four  tasks: 

(1)  strategic  and  tactical  intelligence,  (2)  technical 
intelligence,  (3)  system  operational  characteristics,  and 
(4)  training. 

13.1  Strategic  and  tactical  intelligence  OIR:  5 

A surveillance  system  exists  to  obtain  strategic 
and  tactical  intelligence,  which  is  sent  to  the 
user  agencies.  However,  this  information  should 
also  be  "learned"  by  the  system,  by  feedback  to  the 
system  memory  function,  so  as  to  improve  surveillance 
system  performance  as  new  information  is  acquired. 

13.2  Technical  intelligence OIR:  5 

The  surveillance  system  acquires  and  provides  user 
agencies  technical  intelligence  regarding  target 
characteristics.  However,  this  intelligence  is  the 
cornerstone  upon  which  successful  system  functioning 
is  built.  Therefore,  new  technical  intelligence 
should  also  be  incorporated  into  the  system  to  en- 
hance functioning.  Any  technical  intelligence  pro- 
vided to  the  system  from  outside  agencies  should  be 
incorporated  as  well. 

13.3  System  operational  characteristics . OIR:  5 

One  cannot  precisely  know  until  a system  is  fully 
operational  all  the  nuances  of  man-machine- task 
interaction  that  will  be  required  for  effective 
system  operation.  As  the  system  operates,  system 
.operational  characteristics  of  a very  detailed 
nature  become  evident.  These  lead  to  understanding 
of  system  strengths  and  weaknesses  and  can  be  uti- 
lized to  enhance  system  performance. 
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13.4  Training 


OIR:  S 


In  adaptive  systems  such  as  man  and  some  machines, 
the  results  of  the  three  preceding  tasks  are  auto- 
matically incorporated  into  the  system  memory  func- 
tion, hopefully  in  a way  to  improve  system  performance. 
However,  both  men  and  adaptive  machines  require  a 
certain  amount  of  training  or  reprogramming  to  ensure 
that  the  right  things  are  incorporated  into  system 
memory  as  a function  of  system  experience. 
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CHAPTER  3 


A MODEL  OF  HOW  MAN  FUNCTIONS 
IN  SURVEILLANCE  SYSTEMS 

There  are  two  fundamentally  different  sources  of  knowl- 
edge concerning  man's  functioning  that  should  be  of  importance 
to  designers  of  surveillance  systems.  The  first  has  to  do 
with  his  observable  performance,  that  is,  how  well  he  actually 
performs  the  tasks  that  are  assigned  to  him.  This  evidence 
takes  the  form  of  such  experimentally  measurable  outputs  as 
level  of  alertness  (vigilance);  target  detection  performance; 
accuracy  of  feature  extraction/association,  target  tracking, 
localization,  and  classification;  communication  skills;  and 
various  interactions  with  the  machine. 

A considerable  body  of  information  is  presented  in  Volume 
II  of  this  report  concerning  the  actual  performance  skills  of 
operators  in  different  types  of  surveillance  systems.  In  this 
volume,  we  will  confine  ourselves  to  a second  source  of  knowl- 
edge concerning  man's  functioning.  This  takes  the  form  cf  a 
model  of  human  information  processing  and  it  will  be  evident 
that  the  operator  performs  as  a kind  of  specialized  surveil- 
lance system  in  his  own  right. 

It  is  emphasized  that  a treatment  of  this  second  kind  of 
unobservable,  internal  behavior  relies  heavily  on  theory  and 
laboratory  experimentation.  Thus,  the  model  which  follows  is 
a distillation  of  some  key  theoretical  concepts  from  psychology 
and  physiology  that  we  feel  are  relevant  to  an  understanding 
of  human  information  processing  in  surveillance  systems.  The 
model  does  not  represent  any  single  viewpoint;  it  is  an  inter- 
pretation and  synthesis  that  will  provide  some  foundation  for 
the  systems  designer  to  apprehend  a subset  of  research  that  is 
relevant  to  human  operators  in  surveillance  environments. 


Some  of  the  components  of  the  model  have  been  well  demonstrated 
in  laboratory  studies,  some  have  been  strongly  implied  from 
laboratory  research  and  through  observations,  and  some  compo- 
nents of  the  model  are  theoretical  constructs  necessary  to 
link  other  components  to  one  another  and  to  explain  certain 
aspects  of  human  performance. 

We  wish  to  emphasize  that  this  is  a functional  model;  the 
components  are  not  necessarily  homologous  with  the  physiological 
mechanisms  that  must  underlie  human  information  processing, 
since  the  exact  mechanisms  underlying  this  kind  of  behavior  are 
not  known  at  this  time. 

The  Model  in  Brief 

Figure  3.1  shows  the  functional  model  of  human  information 
processing.  The  model  posits  that  as  the  stimulus  environment 
is  scanned t images  are  stored  in  temporary  buffers;  features 
are  then  extracted  from  the  sensory  images  before  the  image 
decays.  By  features,  w s mean  a set  of  descriptives  that  in- 
clude, as  well,  the  relationships  among  descriptives. 

The  model  shows  preliminary  processing  in  memory  before 
a given  feature  set  impinges  upon  human  consciousness . The 
so-called  "cocktail  party  phenomenon"  is  an  example  of  this 
preliminary  processing:  While  one's  attention  apparently  may 

be  devoted  to  a single  conversation  during  the  cocktail  party, 
some  rudimentary  parallel  processing  of  other  conversations 
is  occurring,  because  one  can  immediately  switch  channels  when 
his  name  is  heard  coming  from  one  of  the  "unattended  to"  con- 
versations. The  mechanism  for  explaining  this  includes  a 
pertinence  function  which  biases  the  preliminary  processing 
and  attaches  priorities  to  various  stimuli  or  feature  sets. 

The  result  of  the  above  processing  is  that  certain  features 
are  selected  for  greater  attention  and,  conversely,  other  fea- 
tures are  rejected  and  do  not  come  under  scrutiny  of  consciousness 
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An  instantaneous,  working  buffer t which  we  may  call 
nooneoioueneee t"  is  postulated.  In  this  working  buffer,  the 
human  has  great  opportunity  for  exerting  some  sort  of  control 
over  the  other  aspects  of  his  information  processing  system: 

He  may  direct  his  scanning  strategy;  as  we  shall  see  later, 
he  may  instigate  rehearsal  and  other  procedures  for  ensuring 
that  items  are  retained  in  short-term  memory  or  that  they  are 
transferred  to  long-term  memory ; he  may  adjust  his  biases  so 
as  to  affect  pertinence  and  therefore  shift  his  attention; 
and  he  may,  to  some  degree,  direct  his  recognition  and  recall 
activities . 

Closely  linked  with  the  working  buffer  is  a short-term 
memory  store  which  will  retain  information  for  approximately 
30  seconds.  This  storage  provides  the  immediate  and  highly 
interactive  link  between  the  working  buffer  and  memory  that 
is  needed  to  maintain  continuity  with  ongoing  tasks  and  the 
stimulus  environment.  Information  that  is  more  permanently 
stored  resides  in  long- l.erm  memory , and  this  information  is 
not  as  accessible  as  that  in  short-term  memory. 

The  reader  will  recall  that  this  is  a functional  model, 
not  a physiological  one.  This  is  particularly  true  in  the 
case  of  short-term  and  long-term  memory;  the  distinction  is 
a functional  one  and,  although  a large  amount  of  experimental 
evidence  reveals  the  distinction  between  the  characteristics 
of  information  storage  in  short-term  memory  and  that  in  long- 
term memory , it  is  not  at  all  clear  that  these  two  memories 
occupy  different  neurophysiological  locations.  It  may  rather 
be  that  there  are  differences  in  the  organization  and  accessi- 
bility of  different  kinds  of  information  stored  in  a single 
location . 

Whatever  the  physioloigcal  differences  or  similarities 
between  short-term  and  long-term  memory , it  is  clear  that  in- 
formation is  less  accessible  in  long-term  memory.  For  certain 
kinds  of  recognition  tasks  or  the  recall  of  previously  learned 
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information,  it  appears  that  consciousness  creates  a query 
and  associational  search  of  long-term  memory ; and  the  output 
of  the  search  is  subsequently  tested.  If  recognition  has 
indeed  taken  place,  or  the  desired  information  recalled , the 
process  is  terminated  and  consciousness  becomes  aware  of  the 
result.  If  the  search  has  been  unsuccessful,  a new  search 
may  be  initiated  with  the  query  modified  to  take  advantage 
of  any  associations  retrieved  from  the  prior  search. 

This  subsystem  may  be  influenced  by  a function  we  have 
labeled  acceptance  criteria . Often  we  are  confronted  with  a 
partial  pattern  that  we  subsequently  "recognize"  even  though 
it  may  not  contain  all  of  the  features  we  have  learned  to 
associate  with  that  pattern.  For  example,  a two-dimensional, 
black  and  white  photograph  of  a familiar  face  usually  passes 
the  acceptance  criteria  even  though  it  contains  only  a sub- 
set of  the  features  normally  defining  that  face. 

A component  in  the  model  which  directly  and  indirectly 
influences  many  aspects  of  human  information  processing  has 
been  labeled  cognitive  biaeee.  Webster  defines  "cognition" 
as  the  process  of  knowing  or  perceiving.  Psychologists  use 
the  terms  "cognition"  and  "cognitive"  to  refer  to  the  kind  of 
human  information  processing  under  discussion  in  this  document; 
sometimes  "cognition"  is  used  by  psychologists  to  include 
"thinking"  and  other  hard-to-de f ine  topics  like  "awareness." 

We  have  used  the  term,  in  conjunction  with  "bias,"  to  create 
a convenient  mechanism  to  describe  how  past  and  present  expe- 
riences influence  perception  of  current  stimulu*s  inputs  and 
recall  of  past  information  stored  in  memory.  The  cognitive 
bias  mechanism  is  really  a part  of  memory,  but  it  is  extremely 
useful,  to  separate  it  from  our  main  treatment  of  memory  in 
order  to  focus  upon  certain  peculiar  human  phenomena,  to  be 
discussed  later. 

The  final  component  in  the  model  represents  the  overt 
responses  which  may  be  empirically  observed.  With  few 
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exceptions,  most  of  what  is  known  about  the  internal  workings 
of  human  cognition  has  been  inferred  from  studies  where  scien- 
tists manipulate  stimulus  input  to  human  subjects  and  care- 
fully measure  the  output  in  the  form  of  overt  responses.  The 
model  does  not  discuss  the  many  types  of  overt  human  responses, 
but  Volume  II  of  this  report  addresses  those  aspects  of  human 
responses  relevant  to  surveillance  systems. 

Various  elements  of  the  model  will  now  be  described  in 
more  detail  with  specific  comments  on  the  strengths  and  weak- 
nesses of  human  operators  with  respect  to  the  various  functions. 

Memory 

Investigators  in  the  area  of  memory  are  fairly  well  agreed 
that  there  are  three  basic  types  of  memory: 

1.  Temporary  buffers 

2.  Short-term  memory 

3.  Long-term  memory 

The  temporary  buffers  represent  the  first  and  most 
transient  element  of  memory  store.  These  contain  fairly 
literal  representations  of  the  stimulus  environment.  The 
transient  memory  has  been  called  iconic  and  echoic  memory 
for  visual  and  auditory  images  respectively.  That  is  to  say, 
an  icon  or  exact  visual  representation  seems  to  be  stored  in 
this  memory,  and  an  echo  or  exact  auditory  representation 
seems  to  be  stored  in  this  memory. 

The  iconic  memory  subsystem  stores  something  analogous 
to  "snapshot"  of  the  stimulus  pattern.  This  snapshot  decays 
rapidly,  in  about  one  second.  The  operator  can  extract  in- 
formation from  this  decaying  memory  for  as  long  as  it  is 
available.  Echoic  memory  is  also  quite  short.  Various  ex- 
periments have  shown  that  the  echo,  which  has  not  been  exten- 
sively coded  but  is,  rather,  a high  fidelity  representation 
of  the  impression  that  the  physical  stimulus  makes  upon  the 
sensory  system,  persists  for  about  two  to  ten  seconds. 
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There  is  nearly  unanimous  agreement  on  the  existence  of 
two  types  of  relatively  durable  memory  in  human  operators, 
ehort  term  and  long  term.  The  short-term  memory  store  may 
maintain  a given  feature  set  for  about  30  seconds,  and  the 
long-term  memory  store  maintains  information  for  much  longer 
periods  of  time.  Indeed,  somj  investigators  feel  that  long- 
term memory  is  permanent;  however,  interference  from  more 
recent  associations  and  learning  experiences  makes  access  to 
older  material  more  difficult.  Other  investigators  feel  that 
information  may  slowly  decay  from  long-term  memory.  A very 
.few  investigators  feel  that  there  is  no  qual :'.t ative  difference 
between  short-term  and  long-term  memory  and  that  the  two 
memory  systems  actually  represent  the  ends  of  the  continuum 
of  a single-memory  system  thdt  shows  loss  of  information  as 
a result  of  time  and  certain  other  conditions.  This  latter 
viewpoint,  however,  is  not  consistent  with  certain  neurological 
evidence . 


In  addition  to  their  relative  duration,  another  way  of 
distinguishing  between  short-term  and  long-term  memory  is  by 
their  relative  capacities.  The  capacity  of  short-term  memory 
is  approximately  seven  items  where  an  item  may  be  a letter,  a 
number,  a word,  a string  of  numbers,  etc.  The  number  of 
i*'ms  that  can  be  contained  in  short-term  memory  is  surpris- 
ingly constant  although  the  amount  of  information  represented 
by  the  seven  items  may  be  variable  from  an  information  theory 
point  of  view. 

The  capacity  of  long-term  memory,  as  is  intuitively 
obvious,  is  much,  much  larger,  and  the  absolute  capacity,  in 
terms  of  items  of  information  stored  is  exceedingly  large. 

In  contrasting  the  memories  of  men  and  machines,  one 
must  be  impressed  with  the  extraordinary  storage  capacity  of 
human  memory.  It  may  well  be  that  any  item  that  is  passed 
from  short-term  into  long-term  memory  is,  in  some  sense,  per- 
manently stored.  Man's  extremely  rich  memory  provides  him 


43 


with  a capability  for  rapid  recognition  of  many  patterns  of 
information,  including  degraded  patterns,  that  are  important 
for  surveillance  system  operation.  However,  the  efficiency 
of  man's  memory  retrieval  depends  upon  degree  of  initial 
learning,  tht  frequency  with  which  the  information  is  re- 
trieved, and  many  ether  variables.  Whether  or  not  man's 
remory  decays  in  the  sense  that  information  is  actually  lost, 
or  whether  retrieval  from  memory  storage  simply  becomes  more 
time-consuming  and  difficult  when  the  information  is  rarely 
used,  is  a moot  point.  There  seems  little  doubt  man's  memory 
systems  exhibit  both  less  reliable  storage  and  less  reliable 
retrieval  than  does  the  memory  of  machines  and  thus  are  highly 
significant  areas  for  computer  aiding  in  an  optimum  man- 
machine  design. 

Attention 

The  elements  of  attention  as  reflected  in  the  model 
include  the  following  functions: 

1.  Image  storage  in  temporary  buffers 

2.  Feature  extraction 

3.  Preliminary  processing  in  memory 

4.  Selection  of  certain  features  for  greater 
attention 

5.  Pertinence  evaluation 

We  have  remarked  upon  the  function  of  the  temporary 
buffer;  it  contains  a high  fidelity  representation  of  the 
stimulus  environment.  This  representation  decays  very  quickly 
and/or  it  is  overlaid  with  new  material.  During  the  time  in 
which  the  representation  remains  viable,  features  are  extracted 
from  it.  Undoubtedly  this  feature  code  is  complex;  it  repre- 
sents descriptive  features  plus  the  relationships  between 
features  plus  the  relationships  between  relationships,  and  so 
on . 
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The  next  point  in  the  flow  of  information  is  critical  for 
an  adequate  model  of  attention.  All  of  the  features  uhich  are 
extracted  are  subjected  to  some  sort  of  preliminary  processing 
that  discards  most  of  the  features  as  uninteresting  and  retains 
a few  to  be  presented  to  consciousness  as  the  stimuli  for 
immediate  attention.  It  is  perhaps  this  characteristic  of 
human  attention  that  differentiates  it  most  from  comparable 
functions  in  machines.  The  process  is  largely  automatic;  it 
does  not  require  an  act  of  volition.  According  to  the  model, 
certain  features  of  the  stimulus  environment  have  high  per  t- 
neneiee  attached  to  them  and,  during  the  preliminary  proce  ung, 
result  in  attention  being  focused  on  them.  In  other  cases  the 
pertinence  of  a feature  may  or  may  not  be  sufficiently  high  to 
warrant  that  feature  entering  consciousness. 

The  notion  of  differential  pertinence  for  different  fea- 
tures or  a stimulus  pattern  has  obvious  application  in  sur- 
veillance systems.  The  operator  who  efficiently  detects  a 
threat  target  has  a very  high  pertinence  associated  with  cer- 
tain features  of  the  displayed  signal.  A casual  glance  at 
the  information  display  may  result  in  immediate  focus  of 
attention  on  those  features  if  they  are  present.  At  the  same 
time,  he  apparently  d.es  not  attend  to  most  of  the  features 
that  represent  uninteresting  target  signals.  The  pertinenc 
function  provides  more  than  a convenience  in  filtering  out 
certain  features;  it  appears  to  be  an  absolute  necessity,  for 
men  do  not  appear  to  be  capable  of  consciously  attending  to 
all  of  the  features  of  their  immediate  stimulus  environment. 

We  wish  to  digress  at  this  point  to  examine  a popular 
fallacy  in  man-machine  comparisons.  Often  it  is  said 
computer  is  a fast  processor,  while  the  man  is  slow.  It  ta 
true  that  men  are  generally  slow  at  sequential  operations, 
like  arithmetic  computations,  and  computers  can  be  made  to 
perform  these  tasks  extremely  quickly  and  accurately.  However, 
this  is  not  equivalent  to  saying  that  man  is  a slow  processor 
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and  that  computers  are  fast.  If  one  reflects  on  how  man 
his  visual  stimulus  environment  and  immediately  recognizes 
innumerable  patterns  of  information  with  different  levels  of 
pertinence  for  his  purpose  at  the  moment,  it  will  be  clear 
that  he  is  an  exceedingly  fast  processor  of  certain  kinds  of 
information  input.  In  general  he  is  capable  of  processing  an 
extremely  large  number  of  pattern  features  and  using  selective 
attention  to  focus  on  the  relevant  subset  of  features.  In 
this  process  he  efficiently  rejects  irrelevant  features,  he 
is  relatively  undisturbed  by  the  transformations  that  are 
irrelevant  to  identifying  the  nature  of  the  object  attended 
to,  he  is  highly  adaptive  to  changing  signal  characteristics 
from  the  object  of  interest,  and  he  is  capable  of  anticipating 
and  predicting  certain  pattern  features,  given  other  features. 
In  contrast,  machines  are  generally  much  less  adaptive  than 
man  to  changing  signal  and  environmental  characteristics. 

The  machine  is  highly  dependent  on  prior  definitions  of  sig- 
nal characteristics  and  is  usually  strongly  bound  to  previously 
specified  instructions  (although  some  degree  of  adaptation  may 
be  possible).  There  is  perhaps  no  other  single  area  of  dif- 
ference between  men  and  machines  that  so  importantly  affects 
the  critical  processes  associated  with  pattern  recognition, 
feature  extraction,  and  target  classification. 

Recognition  and  Recall 

We  have  discussed  some  of  the  properties  of  long-term 
and  short-term  memory:  how  the  information  is  entered  into 

these  memory  stores,  and  some  of  the  conditions-  under  which 
retrieval  from  these  memory  stores  takes  place.  The  present 
section  focuses  on  the  formal  recognition  and  recall  process, 
which  is  of  considerable  interest  in  surveillance  operations 
as  it  is  the  basis  for  classification  performance  in  humans. 

The  elements  cf  recognition  and  recall  in  the  model  are: 

1.  The  working  buffer  ("consciousness") 

2.  Short-term  memory 
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3. 


Long-term  memory 

4.  Feature  query 

5.  Output  features 

6.  Acceptance  criteria 

The  model  postulates  that  during  attempts  at  recognition 
or  recall,  a query  is  generated.  This  query  embodies  a set 
of  features  immediately  known  by  consciousness  to  be  associated 
with  the  item  to  be  recalled  or  recognized,  for  example,  a 
particular  target  characteristic.  A search  of  long-term  memory 
is  then  initiated.  The  output  of  this  search  is  a list  of  fea- 
tures associated  with  the  query  features  through  prior  experi- 
ence. Different  associations  may  have  different  strengths  and 
lead  to  a weighting  of  the  output  features'  relationship  to 
the  query  features.  For  example,  there  may  be  a weak  associa- 
tion of  the  signal  input  with  certain  classes  of  targets  and 
a stronger  association  with  other  classes. 

A decision  mechanism  tests  the  list  of  output  features 
against  the  conditions  of  the  query.  Was  a classification 
retrieved?  Is  the  classification  correct?  This  latter  ques- 
tion may  be  answered  in  two  ways:  First,  if  the  weight  or 
strength  of  the  association  between  the  query  features  and  the 
classification  is  very  high,  the  classification  may  be  accepted 
immediately.  Second,  if  the  strength  of  the  association  does 
not  pass  the  acceptance  criterion,  a tentative  classification 
may  be  formed  which  comprises  a new  query  that  is  then  sub- 
jected to  a new  associative  search  of  long-te rm.  memory . If 
it  is  the  correct  classification,  this  search  should  produce 
at  least  some  of  the  features  from  the  original  query--that 
is,  some  representation  of  the  target  features. 

If  classification  is  not  retrieved  in  the  first  search, 
several  things  may  happen:  Consciousness  may  decide  that  a 

precise  classification  is  not  necessary,  that  just  associating 
the  signal  with  the  broad  category  "non-threat"  is  sufficient, 


and  that  it  should  go  on  to  other  things.  Or,  if  a more 
specific  classification  is  required,  the  list  of  output  fea- 
tures may  be  appended  to  the  lsit  of  query  features  to  formu- 
late a modified  query,  and  a new  search  of  long-term  memory 
may  be  initiated.  It  is  not  at  all  necessary  that  conscious- 
ness play  an  active  role  in  initiating  each  iteration  of 
this  process.  One  can  easily  recall  instances  where  a recall 
process  has  not  had  a satisfactory  result  immediately,  and 
the  whole  problem  is  "put  out  of  mind"  indefinitely,  often 
with  the  result  that  the  correct  answer  presents  itself  at 
some  later  point. 

The  process  we  have  just  described  may  have  three  out- 
comes: It  may  converge  upon  the  desired  associated  features 

on  one  or  several  passes;  it  may  diverge,  always  coming  up 
with  unacceptable  outputs;  or  it  may  loop,  always  coming  up 
with  the  same  but  unacceptable  answer. 

The  system,  as  a whole,  is  not  constrained  to  dealing 
only  with  the  associations  resulting  from  searches  of  long- 
term memory.  The  results  of  the  search  of  long-term  memory 
may  provide  features  or  cues  that  will  stimulate  a new 
scanning  strategy  for  acquiring  additional  features  to  be 
used  in  future  modification  of  the  query. 

We  have  already  commented  upon  the  richness  of  man's 
memory  and  some  of  the  difficulties  he  frequently  experiences 
in  rapidly  extracting  the  desired  information  from  it.  Be- 
cause recognition  and  recall  are  intimately  linked  with  the 
memory  function  it  is  inevitable  that  man's  functioning  in 
these  areas  should  be  extraordinarily  good  in  some  respects 
(e.g.,  the  nearly  instantaneous  recognition  on  the  telephone 
of  the  voice  of  an  old  friend  that  one  has  not  heard  from  for 
a long  time),  but,  for  some  kinds  of  information,  be  subject 
to  the  deficiencie-  associated  with  slow  and  unreliable  re- 
trieval. It  seems  that  man's  recognition  and  recall  of  highly 
complex  stimulus  patterns  is  quite  efficient  but  that  his 
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recall  of  detailed  facts  may  be  cumbersome  and  subject  to 
degradation  with  time. 

A final,  and  most  important,  feature  of  man's  recognition 
and  recall  is  his  ability  to  rapidly  generate  new  hypotheses 
about  stimulus  patterns  and  new  tests  for  evaluating  these 
hypotheses.  This  is  a capability  that  we  find  to  be  quite 
limited  in  machines  and  another  characteristic  of  man  that 
presently  necessitates  his  continuing  role  made  in  the  opera- 
tion of  complex  surveillance  systems. 

Consciousness 

We  have  placed  consciousness  in  the  model  in  a central 
position  as  the  primary  working  buffer  for  human  information 
processing.  In  a sense,  consciousness  is  analogous  to  the 
working  registers  of  a simple  serial  processor  computer. 
Consciousness  receives  the  output  of  the  selective  attention 
mechanism;  it  initiates  certain  searches  of  long-term  memory, 
certain  changes  in  cognitive  biases,  and,  eventually,  certain 
overt  responses.  Consciousness  is  the  prime  mover  in  decision 
making,  a topic  of  considerable  theoretical  and  experimental 
attention.  It  is  slavishly  dependent  on  input  from  memory, 
indirectly  through  the  effects  of  memory  upon  the  attention 
mechanism  and  cognitive  biases,  and  directly  through  the  out- 
put of  associational  searches.  It  might  be  said  that  conscious 
awareness  is  only  moderately  dependent  upon  the  physical  prop- 
erties of  the  stimulus  environment,  for  the  transmission  of 
those  properties  to  consciousness  is  a function  of  coding 
transformation,  filtering,  and  distortion  resulting  from  the 
effects  of  the  memory  system  upon  the  perceptual  system.  And 
memory  is  not  under  direct  control  of  consciousness. 

We  must  warn  the  reader  that  we  have  taken  a straightfor- 
ward, simplistic  approach  to  the  treatment  of  consciousness. 
There  is  a paucity  of  experimental  data  on  the  topic  and  there- 
fore few  guidelines  for  incorporating  it  into  a model  of  infor- 
mation processing.  However,  it  is  explicit  or  implicit  in  a 
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very  large  number  of  theories  and  approaches  to  the  general 
human  cognitive  process  and  the  model  certainly  seems  incomplete 
without  it. 

Cognitive  Biases 

This  component  was  included  in  the  model  to  represent 
the  special  effects  of  memory  upon  certain  aspects  of  human 
information  processing..  T is  is  a somewhat  artificial  com- 
ponent because  a "black  boA1'  containing  cognitive  biases  in 
no  way  exists  as  such  the  fi  ow  of  events  representing  in- 
formation processing.  Rather,  these  cognitive  biases  are  the 
functional  results  of  past  events,  as  they  have  been  preserved 
in  the  memory  stores.  We  feel,  however,  that  it  is  convenient 
to  distinguish  the  effects  of  cognitive  biases  from  the  role 
that  memory  plays  in  providing  recall  and  recognition  facilities. 
Also,  by  somewhat  artificially  distinguishing  the  effects  of 
memory  in  the  form  of  cognitive  biases  via  a separate  component 
in  the  model,  we  have  provided  a reminder  to  system  designers 
of  certain  important,  if  somewhat  peculiar  and  disconcerting, 
informati  1 nrocessing  characteristics  of  the  human  system. 

Cognitive  biases  have  two  important  influences  on  human 
information  processing.  They  actually  control,  modify,  and 
distort  our  more  or  less  instantaneous  perception  of  the 
psychological  present,  and,  to  an  even  greater  degree,  they 
control,  modify,  and  distort  material  that  is  retrieved  from 
long-term  memory,  the  psychological  past. 

Cognitive  biases  are  themselves  a result  of  material  that 
is  stored  in  long-term  memory;  in  some  cases  the  cognitive 
biases  may  be  traced  to  a single  past  event,  such  as  certain 
kinds  of  instructions  received  by  an  operator  prior  to  standing 
watch.  Or,  the  formation  of  cognitive  biases  may  be  the  result 
of  a large  number  of  events  spanning  several  years,  as  in  the 
case  of  perceptual  distortion  due  to  racial  prejudices. 

The  manner  in  which  cognitive  biases  effect  our  percep- 
tion of  things  is  not  precisely  known.  However,  at  least  two 


possible  explanations  have  been  put  forth,  Haber  (1966)  showed 
that  at  least  some  phenomena  due  to  cognitive  biases  can  be 
caused  by  distortions  in  encoding  the  stimuli  for  later  pro- 
cessing. In  terms  of  the  model,  this  would  mean  that  a cogni- 
tive lias  would  cause  a high  pertinence  to  be  given  to  certain 
feature  sets  stored  in  memory.  As  new  features  are  extracted 
from  images  in  the  temporary  buffer,  they  are  subjected  to 
preliminary  processing  in  memory  to  determine  which  will  be 
selected  for  greater  attention.  At  this  point  an  error  occurs-- 
the  output  from  the  preliminary  processing  includes  some  fea- 
tures which  received  the  high  pertinence,  but  which  were  not 
part  of  the  set  of  features  extracted  from  the  image. 

The  mechanism  to  account  for  this  kind  of  error  was 
alluded  to  earlier.  Increases  in  pertinence  are  functionally 
equivalent  to  decreases  in  the  threshold  for  perceiving  fea- 
tures. A feature  set  with  a high  pertinence  may  be  so  sensi- 
tized that  it  will  be  triggered  by  noise  input,  or  more  likely, 
by  a feature  set  bearing  some  similarities  to  the  sought-after 
features  . 

A second  explanation  of  the  apparent  effects  of  cognitive 
biases  is  that  there  is  no  effect  on  the  perceptual  process, 
but  rather,  the  reporting  or  response  process  is  modified  by 
erroneous  memories  of  what  was  perceived.  In  other  words, 
the  person  perceives  correctly  but  remembers  incorrectly  at 
the  time  of  reporting  his  perceptions. 

In  any  event,  an  important  effect  of  cognitive  biases  is 
in  their  contribution  to  performance  variability  between  hu- 
mans, and  to  variability  within  performances  of  a single  opera- 
tor on  different  occasions.  In  general,  the  mere  highly 
controlled  and  structured  a task  is,  the  less  opportunity 
there  is  for  variability  due  to  cognitive  bias.  In  some  cases, 
it  may  be  possible  to  achieve  very  similar  cognitive  biases 
within  the  individuals  of  a group.  An  example  of  this  is  pro- 
vided by  the  very  small  variability  among  surveillance  system 
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operators  in  signal  detection  performance  under  the  alerted 
conditions  that  usually  prevail  when  the  recognition  differ- 
tial  of  the  system  is  being  measured.  The  conditions  of  such 
tests  produce  a similar  cognitive  bias  in  all  operators  in  the 
form  of  high  expectation  for  the  imminent  appearance  of  a sig- 
nal on  their  display.  This  situation  is  to  be  contrasted 
with  more  typical  operations  in  many  surveillance  systems 
where  there  is  little  control  over  the  alertness  of  operators 
or  over  other  cognitive  biases  that  they  may  have.  The  less 
structured  task  environment  characteristic  of  routine  opera- 
tions allows  much  greater  leeway  for  the  effects  of  cognitive 
biases  to  exercise  themselves  differentially  among  individuals. 
Different  biases  govern  the  manner  in  which  attention  is  devoted 
to  the  displays,  the  rigor  with  which  recall  and  recognition 
are  executed,  the  operator’s  perceptual  "set,"  and  so  on. 

Cognitive  biases  appear  to  be  a characteristic  of  man  for 
which  there  is  no  obvious  counterpart  in  machines.  The  central 
problem  is  not  so  much  that  a bias  may  exist,  but  that  it  is 
difficult  for  others  to  know  that  it  exists  and  in  what  form 
it  exists.  Since  cognitive  biases  are  a derivative  of  long- 
term memory,  they  can  only  be  changed  through  appropriate 
retraining.  It  has  been  demonstrated  that  this  can  be  done 
within  the  context  of  surveillance  operations  (Mecherikoff , 

1974)  but  the  problem  remains  that  the  cognitive  biases  of 
most  operators  remain  unknown  and  therefore  constitute  a 
significant  source  of  operator  differences  in  performance. 

Scanning  Strategy 

The  scanning  strategy  component  of  the  model,  like  that 
for  cognitive  biases,  is  a convenient  invention  that  has  func- 
tional utility  but  is  not  homologous  with  any  physiological 
mechanism  in  the  flow  of  information  in  the  human  system. 

We  make  two  fairly  simple  observations  in  connection 
with  scanning.  First,  while  consciousness  may  be  (1)  the 
recipient  of  only  partial  information  from  the  selective 
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attention  process,  (2)  the  dependent  of  a not  totally  trust- 
worthy long-term  memory,  and  (3)  the  occasional  dupe  of  cog- 
nitive biases,  at  least  consciousness  has  some  autonomy  in 
deciding  what  it  will  look  at  or  listen  to.  Thus  it  has  a 
direct  input  to  the  scanning  strategy. 

The  second  observation  is  that  conscious  control  over 
the  scanning  strategy  is  not  complete.  To  some  degree  the 
perceptual  process,  via  selective  attention,  guides  what  we 
see  and  what  we  hear.  Thus  an  operator  in  scanning  a visual 
display  may  detect  a curved  line  in  a group  of  straight  ones. 
He  need  not  consciously  decide  to  make  his  scan  follow  the 
excursion  of  the  curve;  it  seems  that  his  perceptual  process 
will  simply  do  this  for  him  "automatically." 

There  are  many  elements  of  scanning  strategy  that  appear 
to  differentiate  between  men  and  machines.  First,  machines 
are  completely  systematic  scanners  (or  can  be  made  to  be  so) 
while  man  is  a relatively  non-systematic  scanner.  Second, 
the  programming  of  machines  to  follow  a wide  variety  of  un- 
predictable dynamic  changes  in  various  signal  characteristics 
(non-stationarity)  makes  it  difficult  and  expensive  in  terms 
of  computer  capacity  to  develop  scanning  strategies  that  are 
sufficiently  flexible  for  all  possible  signal  variations. 

Man  has  little  difficulty  in  maintaining  this  flexibility, 
and  it  is  one  of  his  most  important  assets  relating  to  feature 
extraction,  feature  association,  and  target  classification. 

Man's  principal  deficiency  in  this  area,  and  it  is  an 
important  one,  is  that  he  does  not  systematically  scan  the 
entire  signal  space  that  is  presented  to  him.  Further,  he  is 
subject  to  degradation  in  his  scanning  behavior  as  a function 
of  monotonous,  routine,  watchstanding  conditions.  In  this 
respect  he  is  almost  certainly  inferior  to  machines  and  a sys- 
tem designer,  who  is  concerned  with  the  man-machine  interface, 
must  devote  a considerable  amount  of  his  resources  to  this 
problem.  Displays  wit.h  extended  signal  histories,  the  use 
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of  artificial  signal  injection,  and  the  sequential  illumination 
of  different  parts  of  the  display  field  are  a few  examples  of 
"countermeasures"  for  this  deficiency  in  man. 


CHAPTER  4 


THE  VARIABLE  OUTPUT  OF  HUMAN  INFORMATION  PROCESSING 
IN  SURVEILLANCE  SYSTEMS 

In  the  previous  chapter,  we  presented  a general  model  of 
man  as  an  information  processor.  If  that  model  described  a 
machine  instead,  we  would  expect  its  various  outputs  (overt 
responses)  related  to  surveillance  operations  to  be  highly 
consistent,  if  not  always  correct.  The  hardware  and  software 
representing  the  various  functions  described  in  the  model 
should  produce  either  identical  response  outputs  to  a given 
signal  input  or  at  least  highly  similar  ones.  Performance 
should  be  much  the  same,  day  in  and  day  out  (barring  cata- 
strophic failure),  and  the  hardware  and  software  components 
in  one  production  line  unit  should,  except  for  deliberate 
modification,  be  interchangeable  with  other  production  line 
units . 

When  man  is  the  subject  of  the  model,  however,  we  are 
confronted  with  a very  different  outcome.  Such  functions  as 
long-term  memory,  cognitive  biases , pertinence , and  acceptance 
criteria  are  conditioned  by  a large  number  of  influences  that 
introduce  extensive  variability  in  the  overt  responses  of  hu- 
man operators.  The  result  is  that  these  elements  of  the  sys- 
tem can  (and  usually  do)  function  quite  differently  in 
different  operators  for  reasons  that  will  be  discussed  shortly. 
Thus,  one  of  the  di’emmas  facing  system  design  and  test  engi- 
neers is  that  the  human  components  of  surveillance  systems, 
unlike  other  components  built  to  a set  of  engineering  specifi- 
cations, are  not  interchangeable  parts.  Not  only  does  the 
output  of  different  operators  differ,  but  the  responses  of 
even  the  same  operator  may  be  inconsistent  from  one  operating 
period  to  the  next.  Indeed,  operators  have  the  capacity  to 
contribute  more  variance  to  total  system  performance  than  any 
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other  component  of  the  system.  Recognition  of  this  fact  has 
probably  been  one  of  the  principal  incentives  to  automation 
in  surveillance  systems.  In  fact,  if  any  other  element  of  the 
system  contributed  as  much  performance  variance  as  the  opera- 
tors do,  there  would  be  immediate  and  intensive  effort  directed 
to  the  problem  of  increasing  the  reliability  of  that  component. 

Sources  of  Operator  Performance  Variability 

The  problem  of  operator  performance  -'ari  ab  i 1 i t y is  not 
so  much  the  fact  that  it  exists  but  that  it  can  be  so  large. 

It  is  large  despite  the  fact  that  both  selection  and  training 
programs  are  clearly  designed  to  minimize  it.  Figure  4.1  illus- 
trates four  major  sources  of  operator  variability;  we  will 
discuss  each  in  turn. 

Innate  abilities.  The  selection  tests  employed  by  the 
Armed  Forces  as  criteria  foi  admittance  to  training  as  sur- 
veillance system  technicians  are  designed  to  homogenize  and 
optimize  the  human  perceptual  and  cognitive  abilities  that 
are  important  to  performance  in  these  systems.  They  are  also 
designed  to  maximize  the  probability  that  the  candidate  will 
benefit  from  (i.e.,  successfully  master)  the  training  curric- 
ulum. The  way  in  which  this  is  done  is  to  set  minimum  cutoff 
scores  on  various  aptitude  tests  so  as  to  effectively  narrow 
the  range  of  individual  differences  in  innate  abilities  that 
are  presumably  relevant  to  job  performance.  How  effective 
these  cutoff  scores  are,  of  course,  depends  upon  the  validity 
of  the  tests  for  predicting  actual  performance  on  the  job. 
Although  test  validity  is  usually  fairly  well  established  with 
respect  to  performance  in  basic  training  schools  t their  pre- 
dictive relationship  to  operational  performance  is  often  a 
matter  that  is  less  clearly  established,  partly  because  objec- 
tive criteria  of  performance  on  the  job  which  test  valida- 
tion can  take  place  typically  are  unavailable. 


Training.  Training  in  a technical  school  is  also  designed 
to  produce  interchangeable  human  system  components.  The  assumption 
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Figure  4.1.  Showing  increasing  performance  variability 
as  a function  of  innate  abilities,  training,  opera- 
tional experience,  and  motivation. 
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is  that  anyone  who  has  successfully  completed  a prescribed 
course  of  training  in  the  operation  of  a particular  system  is 
generally  qualified  to  operate  that  system,  or  soon  will  be, 
given  a modicum  of  on-the-job  experience.  Explicit  in  the 
training  curriculum  is  the  objective  that  all  students  will 
learn  to  operate  the  system  to  some  (usually  unspecified) 
criterion  of  excellence  with  the  hope  that  theoretical  system 
capability  will  be  achieved  in  practice.  For  some  comparatively 
easily  defined  tasks  where  objective  criteria  of  skill  training 
can  readily  be  established  (for  example,  ability  to  interpret 
signals  sent  in  Morse  code  with  specified  s teed  and  accuracy) , 
training  doee  produce  a relatively  homogeneous  product  that 
can  be  plugged  into  an  operational  system  and  which,  for  a 
time  at  least,  results  in  reasonable  uniformity  in  systems 
performance.  Even  where  such  objective  criteria  are  clearly 
specifiable,  however,  individual  differences  in  performance 
soon  begin  to  emerge.  For  example,  within  a few  weeks  after 
graduation  some  operators  will  no  longer  be  able  to  perform 
at  the  school  criterion  level,  some  will  still  perform  at 
about  that  same  level,  and  still  others  will  be  able  to  per- 
form with  significantly  increased  speed  and  accuracy.  Similar 
divergencies  in  performance  certainly  occur  in  the  performance 
of  the  many  complex  tasks  associated  with  the  operation  of 
surveillance  systems,  as  will  be  clear  from  data  presented  in 
Volume  II  of  this  report.  These  performance  differences  re- 
flect the  fact  that  the  selection  criteria  did  not  ensure  that 
all  personnel  assigned  to  the  school  are  equally  able  (or 
willing)  to  benefit  from  the  draining  program.  If  the  selec- 
tion test  scores  were  made  sufficiently  stringent  so  as  to 
produce  "identical"  inputs  to  the  operator  training  program, 
the  resulting  subset  of  personnel  would  likely  be  too  small 
to  meet  operational  requirements.  Although  the  required  caliber 
of  personnel  selected  to  be  surveillance  system  operators  is 
generally  high,  such  peiaonnel  are  in  short  supply;  ideal  se- 
lection criteria  therefore  must  be  compromised,  with  the 
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result  that  some  unwanted  individual  differences  are  intro- 
duced into  the  system. 


Thus,  the  stage  is  set  for  the  development  of  large  indi- 
vidual differences  in  performance.  In  spite  of  the  fact  that 
both  selection  and  training  programs  are  in  some  sense  designed 
to  produce  interchangeable  parts,  the  inputs  to  the  training 
"pipeline"  are  different  in  various  ways  to  begin  with  and 
these  differences  are  magnified  by  virtue  of  differential 
abilities  to  profit  from  the  training  experience.  It  is  impor- 
tant to  note,  however,  that  the  personnel  assignment  system 
operates  as  if  it  had  produced  a batch  of  interchangeable 
parts.  And,  to  a very  real  extent,  the  degree  of  control 
over  the  performance  of  the  human  component  has  been  much 
greater  to  this  point  than  it  over  will  be  again. 

Operational  experience . Once  an  operator  has  been  as- 
signed for  duty  in  an  operating  system,  a third  major  variable 
contributing  to  individual  differences  in  performance  begins 
to  show  its  effects.  We  have  called  this  variable  "operational 
experience,"  as  if  all  operational  experience  were  equally 
meaningful.  But  there  are  a number  of  dimensions  of  experience 
that  have  quite  different  implications  for  operator  performance. 
In  the  case  of  operators  of  surveillance  systems,  certainly 
the  most  relevant  and  powerful  determiner  of  individual  dif- 
ferences in  performance  is  the  frequency  and  recency  with 
which  the  operator  has  had  the  opportunity  to  detect  and 
analyze  "targets  of  interest"  (threat  or  otherwise  reportable 
targets).  In  most  systems,  targets  of  interest  will  be  threat 
targets  as  opposed  to  non-threat  types,  although  in  some  sur- 
veillance systems  safety  considerations  may  make  it  almost  as 
important  to  quickly  and  accurately  recognize  all  types  of 
traffic. 

Major  sources  of  variety  in  operator  experience  have  to 
do  with  the  type  of  signals  encountered  as  a result  of  sensor 
location;  the  type  of  mission  assigned;  procedural  variables 
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associated  with  individual  commands,  particularly  command 
attitude  toward  the  importance  of  routine  target  reporting; 
the  frequency  and  validity  of  performance  feedback  received 
by  the  operators;  and  the  adequacy  of  on-the-job  training. 

This  variety  of  possible  operational  experiences  serves 
further  to  increase  performance  differences  among  operators 
thjt  already  exist  as  a result  of  training  differences  and 
differences  in  native  abilities. 

Motivation.  The  fourth  major  contributor  to  individual 
differences  in  performance  is  motivation,  that  most  important 
and  perhaps  most  difficult  to  measure  of  all  human  traits. 

The  consequences  of  motivation  interact  with  the  three  variables 
already  discussed.  The  impact  of  differential  motivation  at 
both  the  selection  and  training  stages  is  well-known,  serving 
negatively  in  some  cases  to  produce  under-achievement,  and 
positively  in  others  to  produce  performance  well  in  excess  of 
that  expected.  A most  important  motivational  consideration 
in  surveillance  systems  has  to  do  with  the  operator's  percep- 
tion of  the  importance  of  his  (frequently)  very  dull  job  and 
the  extent  to  which  he  is  able  to  maintain  a high  level  of 
performance  in  the  face  of  low  expectancy  for  targets  of 
interest . 

It  is  important  to  note  again  that  even  after  operators 
have  had  various  kinds  of  experience,  personnel  assignments 
continue  to  be  made  as  if  the  human  components  of  operating 
systems  are,  generally  speaking,  interchangeable.  Operators 
of  a given  rank  and  holding  a particular  occupational  specialty 
code  are  treated  as  though  they  indeed  have  equivalent  skills. 
Although  advancement  in  rating  procedures  has  been  established 
that,  through  written  tests  and  practical  demonstrations, 
attempt  to  ensure  that  an  operator  being  promoted  to  a par- 
ticular rank  meets  certain  specified  standards  of  performance, 
the  relevance  of  these  evalustive  procedures  to  actual  ability 
to  perform  the  job  is  sufficiently  uncertain  as  to  leave  room 
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for  large  individual  differences  in  performance  skill  among 
personnel  of  the  same  rank  and  specialty  code. 

At  this  point  it  is  fair  to  ask,  "How  large  are  these 
differences  in  terms  of  meaningful  operational  criteria  and 
how  important  are  they  to  a designer  or  evaluator  of  a sur- 
veillance system?"  A considerable  body  of  objective  data  now 
exists  concerning  individual  differences  in  many  aspects  of 
operator  performance  related  to  surveillance  systems  although 
it  may  not  always  be  in  a form  that  will  be  directly  useful 
to  the  design  engineer.  The  nature  of  the  data  varies  with 
the  complexity  of  the  behavior  involved.  For  example,  there 
are  extensive  data  concerning  simple  psychophysical  relation- 
ships such  as  the  response  of  the  eye  to  various  intensities 
and  wavelengths  of  light,  the  di s c ri min ab i 1 i ty  of  symbols  of 
various  sizes  and  shapes,  and  so  forth.  Extensive  information 
is  also  available  concerning  the  response  of  the  ear  to  vari- 
our  sound  intensities  at  different  frequencies,  the  discrimi- 
nability  of  signals  differing  in  pitch  by  various  amounts, 
and  the  ability  to  detect  signals  masked  by  background  noise. 

Not  only  are  these  relationships  adequately  described  in  various 
human  engineering  guides  (e.g.,  VanCott  & Kinkade,  1972),  but 
the  variability  of  human  performance  for  these  relatively 
simple  psychophysical  tasks  is  comparatively  small.  Thus, 
if  we  are  concerned  only  about  such  relatively  simple  func- 
tions, the  operators  of  surveillance  systems  possibly  can  be 
regarded  as  interchangeable  parts,  assuming  they  meet  some 
minimal  criteria  of  sensitivity. 

The  problem  arises  when  one  proceeds  to  the  more  complex 
human  reactions  associated  with  such  operational  requirements 
as  feature  extraction,  feature  association,  and  target  classi- 
fication. In  each  of  these  functions,  individual  differences 
in  native  abilities,  assimilation  of  training,  operational 
experience,  and  motivation  combine  to  produce  large  individual 
differences  in  operator  performance  skills.  At  the  very  least. 


it  is  imperative  that  the  system  designer  by  aware  of  the 
extent  of  these  individual  differences  and,  if  possible,  the 
specific  reasons  for  them.  Ideally,  he  will  design  his  system 
so  as  to  minimize  the  effects  of  these  differences  on  system 
performance.  This  almost  certainly  means  that  the  focus  of 
his  design  at  the  man-machine-task  level  should  be  on  those 
areas  where  human  performance  is  kncwn  to  be  deficient  or  where 
operator  variability  is  known  to  be  large.  Some  or  these 
areas  are  now  well-known.  For  instance,  the  value  of  large- 
scale  information  storage  and  rapid  retrieval  systems,  in 
aiding  the  operator's  memory  and  ability  to  process  heavy 
information  loads,  has  clearly  been  demonstrated  for  surveil- 
lance systems;  the  development  of  automatic  fault  location 
systems  is  quite  a different  example,  showing  how  computers 
can  be  used  to  minimize  the  effects  of  large  individual  dif- 
ferences in  troubleshooting  skills. 

At  the  same  time,  the  designer  should  be  aware  of  the 
capabilities  of  highly  competent  operators,  that  is,  cases  of 
exceptional  performance  which  can  possibly  serve  as  models  for 
software  design.  From  a programmer's  point  of  view,  these 
are  (1)  unusually  effective  capabilities  that  (nearly  all) 
humans  possess,  and  (2)  capabilities  that  unusually  effective 
humans  possess.  Since  extraordinary  operators,  by  definition, 
cannot  routinely  be  furnished  to  operational  systems,  it 
should  be  one  objective  of  the  design  engineer  to  develop 
software  that  emulates  or  otherwise  at  least  equals  the  per- 
formance of  the  most  highly  competent  operators,  thus  achiev- 
ing the  dual  objective  of  maximizing  the  average  level  of 
system  performance  and  minimizing  its  variability  due  to 
operator  differences. 

If  the  extraordinary  operator  is  to  be  a model  for  future 
system  design,  it  behooves  the  engineering  psychologist  to 
identify  how  that  extraordinary  performance  comes  about  and, 
hopefully,  to  describe  it  in  terms  that  are  meaningful  to 
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systems  designers.  The  first  step  is  to  describe  the  extent 
of  individual  differences  in  actual  operating  performance. 
Fortunately,  there  is  considerable  evidence  concerning  the 
magnitude  of  these  differences  and  some  of  the  variables 
associated  with  them.  Evidence  of  this  kind  with  respect  to 
surveillance  systems  is  presented  in  Volume  II  of  this  report. 

Design  Consequences  of  Individual  Differences  Among  Operators 

The  consequences  of  individual  differences  in  operator 
performance  for  system  design  and  system  test  are  summarized 
below.  These  conclusions  are  generally  supported  by  detailed 
data  presented  in  Volume  II. 

1.  The  operators  of  surveillance  systems  are  not 
equivalent  or  "interchangeable  parts,"  though 
the  personnel  system  generally  operates  as  if 
they  were. 

2.  Operators  are  likely  to  contribute  more  vari- 
ance to  systems  performance  than  hardware  (or 
software)  components. 

3.  Command  attention,  procedural  rules,  reporting 
criteria,  and  "expectancy"  for  threat  targets 
arc  variables  that  strongly  influence  operator 
performance,  both  with  respect  to  target  de- 
tection efficiency  and  reporting  thresholds. 

4.  Selection,  training,  experience,  and  motiva- 
tional variables  interact  so  as  to  magnify 
differences  in  operator  performance  skills. 

5.  One  objective  of  the  system  designer  should  be 
to  minimize  the  impact  on  systems  performance 
of  individual  differences  among  operators;  the 
principal  focus,  of  course,  should  be  -on  the 
areas  of  greatest  human  deficiency. 

6.  The  performance  characteristics  of  superior 
operators  should  serve  as  standards  of  com- 
parison for  automated  system  functions. 

7.  Individual  differences  in  comparatively  simple 
psychophysical  tasks  (e.g.,  alerted  signal  de- 
tection) are  small;  for  complex  tasks  they 
increase  as  task  complexity  increases  (e.g., 
target  classification). 
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Individual  differences  in  ability  to  volun- 
tarily maintain  a high  state  of  alertness 
are  large. 

Operators  differ  extensively  both  in  ability 
to  extract  target  features  from  the  signal 
pattern  and  in  their  ability  to  relate  them 
to  the  nature  of  the  target  in  a logical, 
probabilistic  manner. 

Operators  appear  to  be  more  likely  to  produce 
false  dismissals  than  they  are  false  alarms, 
whereas  the  opposite  may  be  true  of  computer 
algorithms.  This  may  be  a result  of  the 
penalty  for  false  alarms  perceived  by  operators 
as  a result  ot  the  higher  level  review  neces- 
sitated by  contact  reports. 

The  relationship  between  an  operator's  rank 
and  his  proficiency  in  target  detection,  fea- 
ture extraction,  and  classification  is  posi- 
tive but  small.  Frequency  and  recency  of 
experience  with  particular  types  of  targets 
are  considerably  more  important. 

Operator  performance  in  surveillance  systems 
suffers  from  lack  of  feedback  concerning 
missed  opportunities  and  incorrect  classifi- 
cations . 


CHAPTER  5 


• L 


measurement  of  Man-Machine  Performance 

IN  SURVEILLANCE  SYSTEMS 

The  methodology  we  envision  for  man-machine  function 
allocation  requires,  in  part,  the  empirical  evaluation  of  man- 
machine  performance  in  system  configurations  closely  related 
to  those  being  considered  as  design  alternatives.  Once  a 
design  is  realized  as  a prototype,  further  performance  evalua- 
tions will  obviously  be  necessary,  upon  which  the  fate  of  a 
newly  designed  system  may  depend,  and  still  further  evaluations 
can  be  anticipated  throughout  the  life  cycle  of  a system, 
serving,  in  the  end,  as  benchmarks  against  which  later  genera- 
tions of  systems  will  be  compared. 

Therefore,  the  importance  of  adequate  systems  performance 
c/aluations  can  hardly  be.  overestimated,  and  the  presence  of 
man  in  the  system  loop,  whose  variability  we  have  attempted  to 
highlight,  has  extensive  implications  for  the  experimental 
methodology  necessary  to  conduct  adequate  systems  performance 
evaluations.  In  the  course  of  this  study  we  observed  numerous 
methodological  shortcomings  in  operational  evaluations  that 
often  left  the  outcome  of  the  evaluation  very  much  in  doubt, 
despite  very  substantial  investments  in  test  procedures.  In 
many  cases  the  shortcomings  were  associated  with  a failure  to 
consider  vari ab les  associated  with  the  human  operators  in  the 
systems  being  compared. 

A substantial  experimental  methodology  has  collectively 
evolved  from  various  scientific  disciplines  faced  with  the 
problem  of  measuring  relatively  small  effects  in  the  presence 
of  relatively  great  variability.  Since  the  measurement  of 
system  performance  with  men  in  the  loop  usually  poses  just 
such  a problem,  an  extended  discussion  of  this  methodology  is 
contained  in  Volume  II  of  this  report.  However,  the  principal 
points  are  highlighted  below.  Though  many  of  them  are  "obvious," 
they  are  included  here  because  they  are  so  often  violated  in 
operational  tests  of  surveillance  systems. 


Evaluative  tests  of  experimental  man-machine 
systems  require  comparative  data  from  a suit- 
able baseline  (control)  system,  except  in 
rare  instances  where  absolute  criteria  can  be 
specified.  The  necessary  man-machine  perfor- 
mance data  for  the  baseline  system  usually 
must  be  collected  at  the  same  time  data  on  the 
experimental  system  are  collected  because 
suitable  records  of  routine  man-machine  per- 
formance in  operational  systems  are  rarely 
available . 

Comparative  tests  of  experimental  and  currently 
operational  systems  must  take  into  account  op- 
erator and  procedural  variables  that  contribute 
to  total  man-machine  performance,  as  well  as 
differences  in  hardware  or  software. 

Understanding  the  variables  related  to  man- 
machine  strengths  and  weaknesses  in  currently 
operational  systems  is  essential  for  design- 
ing suitable  comparative  tests  of  system 
effecti vene  ss . 

The  set  of  test  signals  used  in  comparative 
system  tests  must  be  selected  so  as  to  be 
representative  of  the  difficulties  posed  at 
each  stage  of  the  functional  taxonomy,  as  well 
as  the  signal  population  of  interest  and  the 
operational  environment. 

Considerations  of  experimental  control  usually 
dictate  the  use  of  synthetic  signals  injected 
into  the  actual  operating  system ; the  parameters 
of  the  injected  signals  must  reflect  the  con- 
siderations outlined  in  No.  4 above. 

The  timing  of  injected  signals  is  particularly 
important  when  the  system  test  involves  ques- 
tions of  operator  scanning  behavior,  vigilance, 
or  expectancy  as  to  the  time  of  appearance  or 
nature  of  the  target. 


Both  men  and  computers  (or  computer  programmers) 
require  comprehensive  sets  of  signals  on  which 
to  learn.  The  discriminations  learned  on  the 
learning  set  must  be  cross - va 1 idat ed  on  an 
independent  test  set  of  representative  signals 
to  properly  assess  the  effectiveness  of  these 
discriminations . 


Individual  differences  among  operating  person- 
nel in  experience,  training,  innate  abilities, 
and  motivational  variables  must  be  taken  into 
account  in  the  design  of  the  operational  test 
and  in  the  analysis  of  the  resulting  data. 

Such  differences  can  otherwise  result  in  mis- 
leading conclusions  about  differences  between 
the  systems  in  man-machine  performance. 

In  the  interest  of  equalizing  or  controlling 
for  individual  differences  among  operators, 
it  is  desirable  to  employ  within-sub ject  test 
designs  in  which  the  same  personnel  operate 
both  the  experimental  and  baseline  systems 
during  the  test.  If  this  is  not  possible, 
personnel  used  in  the  two  conditions  should  be 
matched  as  completely  as  possible  on  all  vari- 
ables known  to  be  relevant  to  performance. 

Negative  attitudes  toward  innovative  systems 
or  procedures  may  develop  and  adversely  affect 
the  operational  test  unless  certain  principles 
of  introduction  are  followed.  A change  advo- 
cate, whose  credentials  are  highly  respected 
by  operational  personnel,  can  be  very  useful 
in  this  respect. 

The  operational  test  should  be  designed  so  as 
to  minimize  atypical  operator  motivations  that 
stem  simpl)  from  the  operator's  knowledge  that 
he  is  participating  in  a test. 

The  operational  test  should  be  designed  so  as 
to  avoid  atypical  operator  "expectancy"  for 
targets  of  special  interest.  However,  this 
objective  may  have  to  be  sacrificed  somewhat 
in  meeting  the  need  for  reliability  of  per- 
formance measurement. 

Usually,  operational  personnel  used  in  system 
tests  should  be  representative  of  "average" 
operators.  The  tendency  to  utilize  superior 
personnel  in  the  experimental  system  should 
be  resisted,  although  some  test  objectives 
might  justify  the  use  of  superior  operators 
in  the  baseline  system. 

When  the  same  test  personnel  operate  both  the 
experimental  and  baseline  systems,  the  test 
design  should  be  such  that  unwanted  performance 
variance  is  not  introduced  by  the  order  in 
which  the  two  systems  are  operated. 


15.  If  the  test  involves  round-the-clock  operations, 
the  test  design  and  data  analysis  should  take 
into  account  the  diurnal  variations  in  the  level 
of  arousal  (alertness)  of  human  operators. 

16.  In  selecting  criterion  measures  of  man-machine 
performance  by  which  the  test  outcomes  will  be 
assessed,  it  is  essential  to  provide  for  re- 
cording of  data  pertinent  to  each  aspect  of  the 
function  taxonomy  that  is  related  to  the  test 
objectives.  The  recording  schemes  used  may 
differ  considerably  for  the  man  and  the  machine. 

17.  Special  signal  sets  may  have  to  be  designed  to 
obtain  appropriate  system  response  measures  on 
some  parts  of  the  function  taxonomy. 

18.  In  the  interest  of  obtaining  reliable  man- 
machine  performance  measures,  substantial  num- 
bers of  measurements  should  be  made  on  similar 
signal  inputs  for  each  part  of  the  function 
taxonomy  under  test. 

General  Conclusions 

There  are  a large  number  of  specific  conclusions  to  be 
found  interspersed  throughout  Volume  II  of  this  report  as  well 
as  those  already  presented  in  this  volume.  However,  we  should 
like  to  emphasize  here  the  broadest  and  most  general  conclu- 
sions that  wc  have  come  to  as  a result  of  this  study. 

When  wc  undertook  this  research,  we  did  it  in  response 
to  a felt  need  for  a systematic  design  feedback  loop  from 
operating  surveillance  systems  to  the  designers  of  future 
systems,  and  we  assumed  that  the  final  results  would  be  usable 
as  guidelines  for  the  design  of  future  surveillance  systems. 
That  seemed  like  a reasonable  objective,  and  wc  hope  the  re- 
sults presented  in  various  parts  of  these  two  volumes  do  prove 
to  be  useful  to  the  surveillance  community;  we  are  reasonably 
certain  that  at  least  some  of  these  materials  will  be.  How- 
ever, the  first  major  conclusion  we  wish  to  present  here  con- 
cerns our  original  expectation  that  we  would  be  able  to 
develop  detailed  design  trade-off  criteria  that  would  be 
applicable  to  the  function  allocation  phase  of  the  design 
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process.  From  this  expectation,  the  reader  can  see  some  impli- 
cation that  we  thought  we  might  develop  a "cookbook"  for  func- 
tion allocation.  We  did  intend  to  proceed  as  far  as  possible 
in  that  direction;  but  we  also  stated  at  the  outset  that  our 
objectives  were  ambitious,  and  we  were  somewhat  dubious  of 
accomplishing  those  objectives  where  the  human  was  involved 
in  the  loop,  because  his  oehavior  is  so  task-specific.  How- 
ever, we  thought  the  machine  side  of  the  equation  might  tend 
to  make  the  problem  more  tractable.  It  now  appears  to  us 
that  the  behavior  of  machines  is  also  highly  task-specific, 
and  therefore  a small  set  of  design  principles  will  not  serve 
to  provide  ready  answers  to  the  much  larger  set  of  specific 
man-machine  function  allocation  problems.  We  have  already 
stated,  and  we  reiterate  as  a major  conclusion,  that  no  "cook- 
book" of  man-machine  function  allocation  recipes  is  impending 
as  a result  of  this  work  or  any  other  we  are  aware  of.  In- 
stead, the  function  allocation  methodology  that  we  have  de- 
scribed must  be  applied  to  individual  function  allocation 
prob  1 ems . 

The  second  major  conclusion  we  have  come  to  is  that  many 
aspects  of  the  detection,  feature  extraction/association,  and 
classification  functions  of  surveillance  systems  cannot  be 
totally  automated,  now,  or  for  some  time  to  come,  without  a 
substantial  sacrifice  in  system  performance.  We  have  con- 
cluued  that  man  must  remain  in  surveillance  systems  as  a part- 
ner in  the  detection  and  po s t - de t ect ion  processing  functions 
for  the  full  performance  potential  of  new  surveillance  s,  stem 
designs  to  be  realized.  Certainly,  we  feel  that  man  should  be 
extensively  machine-aided  in  these  areas,  and  vice  versa.  But 
that  leaves  us  with  a considerable  problem,  because  th''  finz- 
tion  and  task  allocations  that  will  successfully  optimize  this 
man-machine  partnership  remain  to  be  determined.  The  surveil- 
lance community  now  needs  to  undertake  a substantial  and 
dedicated  research  and  development  effort,  hopefully  employing 
to  advantage  some  of  the  material  we  present  in  this  report, 
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to  discover  how  men  and  machines  can  work  most  effectively 
together  in  surveillance  systems. 

A Word  About  Volume  II 

Volume  II  of  this  report  contains  a great  deal  of  addi- 
tional information  that  we  hope  will  be  of  interest  to  indi- 
viduals whose  responsibilities  include  research  and  development 
on  surveillance  systems  hardware,  software,  and  operating 
personnel . 

Following  the  introductory  remarks  in  Chapter  1 of  Volume 
II,  we  present  in  Chapter  2 a more  detailed  description  of 
functions  in  specific  surveillance  systems  and  deal  with  some 
of  the  classified  examples  of  those  functions  which  are  not 
covered  in  the  present  volume. 

In  Chapter  3,  a more  detailed  description  of  human  infor- 
mation processing  in  specific  surveillance  systems  is  provided 
with  examples  of  how  that  processing  occurs  in  the  context  of 
operator  performance  during  surveillance  activities. 

In  Chapter  4,  the  variable  output  of  human  information 
processing  in  surveillance  systems  is  described  in  terms  of 
objective  data  concerning  the  performance  of  operators  in 
specific  surveillance  systems  and  the  variables  that  appear 
to  influence  that  performance. 

In  Chapter  5,  more  detail  is  presented  concerning  the 
considerations  necessary  for  proper  measurement  of  man-machine 
performance  in  surveillance  systems  with  special  attention 
devoted  to  problems  that  arc  unique  to  specific  surveillance 
systems . 

The  appendices  of  Volume  II  are  devoted  to  the  detailed 
reporting  of  a number  of  experiments  directed  at  the  detection 
and  classification  performance  of  operators  in  specific 
surveillance  systems. 
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